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SUPPLEMENTARY  NOTES 


This  report  is  in  the  form  of  a “white  paper,”  which  together  with  its  executive  summary  (AFHRL-TR-78-24)  has 
been  prepared  in  partial  response  to  the  23  June  1977  Office  of  the  Secretary  of  Defense  Memorandum,  subject: 
“Management  Review  of  Maintenance  Training  and  Performance  Aids.”  The  memorandum  was  prepared  as  a result 
of  a formal  management  review  meeting  held  on  1 through  3 February  1977.  Two  major  conclusions  of  that  meeting 
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"v*  This  report  and  its  executive  summary  (AFHRL-TR-78-24)  concern  the  impact  of  improved  maintenance 
guidance  and  information  (IMG&I),  as  well  as  task  oriented  training  (TOT)  technologies  on  DOD  maintenance, 
personnel  and  training  systems.  The  IMG&l  considered  include  fully  proceduralized  job  performance  aid  (FPJPA) 
technology  for  both  non-troubleshooting  (non-TS)  and  troubleshooting  (TS)  tasks;  the  Army  “New  Look"  for 
non-TS  tasks;  and  for  TS  tasks  only:  the  traditional  and  enriched  FORECAST  aid,  traditional  and  enriched  logic  tree 
troubleshooting  aid  (LTTA),  and  traditional  and  Air  Force  symbolic  integrated  mantenance  system  (SIMS),  as  well 
as  a newer  form  of  SIMS  called  the  functionally  oriented  maintenance  manuals  (FOMM). 

However,  the  technologies  or  concepts,  whose  effectiveness  is  supported  by  comparative  hard  data,  include 
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Item  18  Continued: 

ate  addressed  in  the  memo,  namely,  (a)  the  need  to  implement  an  existing  technology  base  in  job  performance  aid 
manuals,  and  (b)  the  need  for  more  effort  on  integrated  technical  data , training,  and  personnel  support  systems  that 
are  based  on  analysis  of  the  job  task  requirements.  The  first  requirement  was  assigned  to  the  Navy  and  fulfilled  by 
NPRDC  TR  77-33,  "Symposium  Proceedings:  Invitational  Conference  on  Status  of  Job  Performance  Aids 
Technology,”  May  1977. 

This  report  and  its  executive  summary,  address  the  second  requirement  which  specifically  addresses  the  requirement 
lor  the  preparation  of  such  a paper  “on  the  applicability  of  job  aids  and  the  impact  on  the  personnel  and  training 
systems.”  Included  in  this  requirement  is  a discussion  of  “the  applicability  of  job  aids,  training  or  both  to  enhance 
military  jobs  proficiency ,'  as  well  as  the  identification  of  service  and  DOD  policies  that  restrict  payoffs  in  this  area. 
(Industrial  approaches  to  task  analysis,  job  aids,  training  and  personnel  utilization  are  also  discussed.) 

The  author  was  designated  as  chairman  of  a committee  which  included  Dr.  Robert  Blanchard  of  the  Navy  Personnel 
Research  and  Development  Center,  San  Diego,  CA;  Mr.  John  K.  Klesch  of  the  Army  Training  Development  Institute, 
Ft  Eustis  VA;  and  Mr.  Harry  Maragides  of  Headquarters,  Marine  Corps,  Washington,  DC.  After  the  first  draft  of  the 
required  paper  was  prepared  a committee  meeting  was  held  at  Headquarters,  Air  Force  Humai  Resources 
Laboratory  (AFHRL),  Brooks  AFB  TX,  on  13  December  1977.  (Although  Mr.  Klesch  was  unable  to  attend,  the 
other  committee  members,  as  well  as  LTC  Roger  Grossel  of  Department  of  Defense  - OASD(MRA&L)WR  and  Mr. 
Robert  Johnson  of  the  Advanced  Systems  Division  (AFHRL),  Wright-Patterson  AFB  OH,  were  present.)  The 
comments  of  this  meeting  resulted  in  some  reorganization  of  the  first  draft,  as  well  as  the  requirement  for  the 
executive  summary.  * 
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only:  the  FPJPA,  FORECAST  aid,  enriched  LTTA,  and  AF  SIMS.  These  hard  data  indicate  that  quality 
implementation  of  any  of  these  technologies  (or  concepts)  will  result  in  more  efficient  performance  of  maintenance 
tasks  than  the  use  of  the  traditional  maintenance  manual  (TMM).  (In  this  regard,  there  are  no  hard  data  which 
indicate  that  the  FOMM  is  more  effective  than  the  AF  SIMS.)  But,  by  far  the  most  dramatic  reductions  of  the  life 
cycle  cost  (LCC)  of  hardware  ownership  can  be  realized  by  the  quality  integrated  application  of  three  of  these  types 
of  IMG&I  with  TOT;  i.e.,  FPJPA,  FORECAST  TS  Aid,  and  enriched  LTTA.  Of  these,  the  FPJPA  has  the  most 
potential. 

Much  of  the  executive  summary  and  the  body  of  the  white  paper  are  similar  in  organization  and  content. 
However,  the  white  paper  contains  a more  complete  treatment  of  human,  hardware,  and  software  problems,  and 
policies  which  are  impeding,  or  will  impede,  the  implementation  of  these  money  saving  technologies.  Although 
dramatic  LCC  reductions  can  be  effected  by  the  quality  implementation  of  1MG&1  and  TOT  technologies,  such 
implementations  require  greater  dollar  investments  in  maintenance  guidance  documents  and  in  training  than  the 
implementing  establishments  are  accustomed  to  spend.  But  additional  money  is  not  enough -quality 
implementations  require  “know  how.”  A mechanism  must  be  developed  which  makes  efficient  and  effective  use  of 
the  “know  how”  of  the  developers  of  the  technologies,  and  makes  them  responsible  and  accountable  for  their  early 
implementations.  In  this  regard,  most  past  implementations  have  “watered  down”  such  technologies. 

Of  special  interest  to  executives  and  managers  are  two  tables  which  consolidate  the  important  attributes  of 
various  IMG&I  types  and  TMM;  a figure  which  summarizes  the  results  of  many  DOD  TOT  studies;  and  a (low 
diagram  wluch  portrays  a model  for  maximizing  benefits  of  FPJPA,  SIMS,  and  TOT  technologies. 

The  white  paper  also  contains  a number  of  appendices  which  summarize  most  of  the  DOD  studies  which  have 
produced  hard  data  concerning  various  IMG&I.  Other  appendices  describe  and  discuss  TMM  and  the  various  IMG&I 
as  well  as  military  and  industrial  applications  of  IMG&I  and  TOT. 
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This  report  represents  a protion  of  the  exploratory  development  program  of  the 
Advanced  Systems  Division,  Air  Force  Human  Resources  Laboratory,  Wright-Patterson 
Air  Force  Base,  Ohio. 

The  preparation  of  this  report  was  documented  under  task  171004,  Job 
Performance  Aids  fo,  Air  Force  Maintenance  Technicians.  The  task  is  part  of  project 
1710,  Training  for  Advanced  Air  Force  Systems.  The  effort  represented  by  this  paper  was 
identified  as  work  unit,  17100427.  Mr.  Robert  Johnson  was  task  scientist.  Dr.  Ross  L. 
Morgan  was  project  scientist. 

The  author  wishes  to  express  his  appreciation  for  the  cooperation  and  inputs  of  the 
other  members  of  the  interservice  committee  designated  for  this  effort.  They  include  Dr. 
Robert  Blanchard  of  the  Navy  Personnel  Research  and  Development  Center,  San  Diego, 
CA;  Mr.  John  K.  Klesch  of  the  Army  Training  Development  Institute,  Ft  Eustis,  VA;and 
Mr.  Harry  Maragides  of  Headquarters,  Marine  Corps,  Washington,  DC.  The  comments  of 
LTC  Roger  Grossel  of  the  Department  of  Defense  - OASD(MRA&L)WR  and  Mr.  Robert 
Johnson  of  the  Advanced  Systems  Division  of  AFHRL  are  also  greatly  appreciated. 
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IMPACT  OF  ADVANCED  MAINTENANCE  DATA  AND  TASK  ORIENTED  TRAINING 
TECHNOLOGIES  ON  MAINTENANCE,  PERSONNEL,  AND  TRAINING  SYSTEMS 


I.  INTRODUCTION 

This  report,  together  with  its  executive  summary  (AFRHL-TR-78-24,  Foley,  1978),  was  prepared  in 
response  to  the  23  June  1977  Office  of  the  Secretary  of  Defense  Memorandum,  subject:  “Management 
Review  of  Maintenance  Training  and  Performance  Aids.”  The  memorandum  was  prepared  as  a result  of  a 
formal  management  review  meeting  held  on  1-3  February  1977. 

Two  major  conclusions  of  that  meeting  are  addressed  in  the  memo,  namely,  (a)  the  need  to 
implement  an  existing  technology  base  in  job  performance  aid  manuals  and  (b)  the  need  for  more  effort  on 
integrated  technical  data , training,  and  personnel  support  systems  that  are  based  on  analysis  of  the  job  task 
requirements.  The  first  need  was  at  least  partially  satisfied  by  a recently  published  report  of  the  Navy 
Personnel  Research  and  Development  Center  concerning  the  status  of  job  performance  aid  (JPA) 
technology  (NPRDC  TR  77-33,  Booher,  1977).  This  paper  concerns  the  second  need  and  specifically 
addresses  the  requirement  of  the  memorandum  for  the  preparation  of  a paper  “on  the  applicability  of  job 
aids  and  the  impact  on  the  personnel  and  training  systems,”  (although  the  appendices  also  contain  a large 
amount  of  materials  relative  to  the  status  of  JPA  technology).  Included  in  the  requirement  for  this  paper  is 
a discussion  of  “the  applicability  of  job  aids,  training  or  both  to  enhance  military  job  proficiency .”  A large 
part  of  the  primary  missions  of  the  Army  and  Marines,  and  almost  all  of  the  Navy  and  Air  Force  primary 
missions,  are  concerned  with  the  development,  operation  and  maintenance  of  complex  hardware  systems; 
therefore,  the  requirement  for  the  enhancement  of  proficiency  in  the  performance  of  operator  and 
maintenance  jobs  is  especially  critical.  Traditionally  the  military  services  have  given  a great  deal  more 
attention  to  the  proficiency  of  operator  task  performance  than  to  the  proficiency  of  maintenance  task 
performance.  But  this  emphasis  is  shifting.  The  life  cycle  costs  (LCCs)  of  such  hardware  systems  are 
extremely  high  and  continue  to  rise.  Important  factors  contributing  to  LCC  are  high  maintenance  personnel 
costs  and  the  high  cost  of  spare  parts,  as  well  as  the  unnecessary  usage  of  spare  parts. 

A large  portion  of  high  maintenance  personnel  costs  in  the  military  services  is  contributed  by  the  cost 
associated  with  first  enlistment  personnel.  Because  of  consistently  low  reenlistment  rates  of  maintenance 
personnel,  a large  portion  of  DOD  maintenance  tasks  must  be  performed  by  relatively  inexperienced  first 
enlistment  personnel.  The  costs  for  supporting  all  first  enlistment  personnel  are  currently  estimated  at 
$15,000  per  person  per  year,  or  $60,000  for  a four-year  enlistment.  But  generally,  the  effective  period  of 
on-the-job  performance  of  maintenance  tasks  for  such  personnel  is  relatively  short  (as  low  as  1 year  in  some 
cases),  making  the  actual  work  year  labor  costs  of  useful  work  from  first  enlistment  personnel  extremely 
high  (as  high  as  $60,000  for  one  year  of  useful  work). 

At  least  two  important  factors  contribute  to  this  generally  short,  on-the-job  performance  time  of  first 
enlistment  maintenance  personnel.  The  entry  training  programs,  both  formal  and  on-the-job,  for  many 
maintenance  personnel  skill  codes  are  or  have  been  long.  (But  in  spite  of  such  lengthy  training  programs, 
there  is  substantial  evidence  that  maintenance  tasks  generally  are  not  performed  with  high  efficiency,  either 
by  first  enlistment  personnel  or  by  personnel  with  longer  on-the-job  experience.)  A second  factor  is  the 
current  “early  out”  policies  of  the  services.  If  the  escalation  of  DOD  maintenance  costs  is  to  be  controlled, 
ways  must  be  found  to  increase  the  on-the-job  utilization  of  first  enlistment  personnel,  as  well  as  to  increase 
the  efficiency  with  which  all  maintenance  tasks  are  performed. 

This  report  is  concerned  with  attaining  these  objectives  by  the  proper  selection  and  implementation 
of  several  available  concepts  and/or  technologies.  One  purpose  is  to  describe  these  concepts  and 
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technologies  which  include  the  job  task  performance  test  (JTPT),  task  oriented  training  (TOT),  and  several 
types  of  maintenance  directions  and/or  information  - such  as  the  fully  proceduralized  job  performance  aid 
(FPJPA),  logic  tree  troubleshooting  aid  (LTTA),  symbolic  integrated  maintenance  system  (SIMS),  and 
functionally  oriented  maintenance  manual  (FOMM).  A second  purpose  is  to  discuss  the  potential  of  each 
concept,  or  technology,  for  reducing  each  cost  factor.  A third  purpose  is  to  present  a general 
implementation  model  or  plan,  which  integrates  compatible  technologies  for  maximum  cost  savings.  A 
fourth  is  to  discuss  approaches  to  task  analysis,  job  aids,  training  and  personnel  utilization  being  applied  by 
industry.  The  final  and  most  important  purpose  is  to  identify  current  practices,  policies,  and  problems 
which  impede  or  prevent  the  implementation  of  such  concepts  and  technologies.  (Since  the  materials  or 
discussions  concerning  the  First,  second,  and  fourth  purposes  are  rather  voluminous,  they  are  only 
summarized  in  the  body  of  this  report.  Where  applicable,  more  extensive  treatments  have  been  included  as 
appendices.) 


II.  IMPROVING  THE  PERSONNEL/HARDWARE  INTERFACE 

The  proper  .application  and  use  of  several  technologies  and/or  concepts  of  improved  maintenance 
guidance  and  information  (1MG&I)  could  improve  the  performance  of  maintenance  tasks.  In  addition,  the 
concurrent  and  integrated  application  of  a few  of  these  maintenance  guidance  and  information  technologies 
(together  with  the  task  oriented  training  technology)  would  greatly  reduce  the  entry  training  time  for  first 
enlistment  maintenance  personnel. 

Improved  Maintenance  Guidance  and  Information 
Technologies  and/or  Concepts 

Table  1 identifies  the  name  and  source  of  each  of  these  technologies  and/or  concepts  together  with  a 
brief  description  of  each.  Most  of  these  technologies  are  aimed  at  improving  the  performance  of 
troubleshooting  (TS)  tasks.  Only  two  are  aimed  at  improving  the  performance  of  non-troubleshooting 
(non-TS)  tasks,  such  as  checkout  tasks;  align,  adjust,  and  calibrate  tasks;  and  install,  remove,  and  replace 
tasks.  Table  2 indicates  these  and  other  attributes  for  the  technologies  and/or  concepts  listed  in  Table  1.  In 
addition,  appendices  A and  B provide  more  complete  descriptive  information  as  well  as  some  historical 
information  concerning  these  technologies  and  concepts. 

Table  2 also  indicates  that  some  of  these  concepts  are  effective  with  highly  trained  and/or 
experienced  personnel,  while  others  will  support  the  performance  of  personnel  with  limited  experience 
and/or  training.  The  effectiveness  of  a few  of  these  concepts  is  supported  by  hard  data,  while  that  of  others 
is  not.  Where  comparative  hard  data  have  been  gathered  for  a concept,  they  have  indicated  that  the  use  of 
the  concept  (or  technology)  by  highly  trained  and  experienced  personnel  would  result  in  more  efficient 
performance  of  the  categories  of  maintenance  covered  (non-TS  or  TS  tasks  or  both)  than  the  use  of  the 
traditional  maintenance  manual  (TMM).  The  key  word  in  this  statement  is  “use.”  An  improved  IMG&I 
concept  or  technology  must  be  used  during  the  performance  of  tasks  to  be  effective. 

The  only  concepts  whose  effectiveness  is  supported  by  hard  data  are  FPJPA,  FORECAST,  enriched 
LTTA,  and  Air  Force  SIMS,  plus  the  TMM.  Most  of  the  studies  providing  these  hard  data  are  summarized  in 
the  various  appendices  of  this  report.  Column  1 of  Table  3 indicates  each  of  these  concepts,  Column  2 
indicates  the  applicable  references  for  each  concept,  and  Column  3 indicates  the  appendices  in  which  these 
references  are  summarized.  All  of  the  studies  indicated  have  produced  some  hard  data.  Column  4 indicates 
those  which  have  produced  substantial  comparative  hard  data  concerning  the  various  concepts.  Indepth 
study  and  analyses  of  these  data  indicate  that  some  IMG&I  technologies  are  more  effective  than  others  for 
high  aptitude  personnel  who  are  highly  trained  and  experienced  but  that  only  FPJPA  and  FORECAST  have 
proven  effectiveness  with  medium  aptitude  personnel  with  limited  training.  The  various  relationships  are 
summarized  in  the  following  paragraphs. 
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Table  1.  Name  and  Brief  Descriptive  Information  Concerning  IMG&I,  and  TMM 


3 Ke>  cd  schematics  and 
keyed  information 
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Table  3.  References  and  Appendices  Containing  Hard  Data 


Concept 

Author  Reference 

Summarized 

in 

Appendix 

Scope  of  Comparison 

FPJPA 

Serendipity,  1969 

D,E 

FPJPA  (non-TS)  vs.  TMM 

Potter  & Thomas,  1976 

D,  G 

FPJPA  (TS)  vs.  LTTA  vs.  TM  M 

Elliott  & Joyce,  1968 
Elliott,  1967a 

D 

FPJPA  vs.  TMM 

Mullen  & Joyce,  1974 

D 

N/A 

Shriver  & Foley,  1975 
Shriver,  1975 

D,  F 

N/A 

Theisen  et  ah,  1978 

11 

N/A 

FORECAST 

Shriver  et  al .,  1964 

C 

FORECAST  vs.  TMM 

Enriched  LTTA 

Potter  & Thomas,  1976 

D,  G 

LTTA  vs.  FPJPA(TS)  vs  TMM 

AF  SIMS 

Serendipity,  1969 

D 

AF  SIMS  vs.  TMM 

Comparative  Effectiveness  of  Various  IMG&I  for  Highly 
Trained  and/or  Experienced  Personnel 

The  comparative  hard  data  produced  by  the  various  studies,  referenced  in  Table  2,  indicate  the 
following  effectiveness  relationships  for  IMG&I  when  used  by  highly  trained  and/or  experienced  personnel. 

1.  For  non-TS  tasks: 

a.  FPJPA »'  TMM 

2.  For  TS  tasks: 

a.  FPJPA  ? Enriched  LTTA  » TMM 

b.  Traditional  FORECAST  » TMM 

c.  AF  SIMS  » TMM 

The  use  of  any  of  the  indicated  improved  technologies  in  lieu  of  the  TMM  would  improve  the 
efficiency  of  highly  trained  and  experienced  maintenance  personnel.  This  improved  efficiency  would  be 
reflected  in  reduced  spare  parts  usage,  shortened  cross-training  on  unfamiliar  hardware,  and  perhaps, 
incieased  numbers  of  maintenance  tasks  performed. 

IMG&I  and  Potential  for  Reducing 
Maintenance  Personnel  Costs 

If  LCC  is  to  be  greatly  reduced,  those  technologies  that  maximize  the  utilization  time  and  efficiency 
of  first  enlistment  personnel  must  be  fully  expfoited.  This  objective  requires  the  application  of  those 
technologies  which  are  not  only  effective  for  improving  maintenance  efficiency  but  also  have  potential  for 
reducing  entry  training  requirements.  Table  2 indicates  that  only  three  technologies,  for  which  hard  data 
are  available,  meet  both  of  these  criteria,  i.e.,  FPJPA,  traditional  FORECAST  and,  by  projection,  enriched 
FORECAST.  The  hard  data  indicate  the  following  effectiveness  relationships: 

1.  For  non-TS  tasks: 

a.  FPJPA  » TMM 

*»  Much  more  effective  than. 

2)  More  effective  than. 
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a.  FPJPA  » TMM 

b.  Traditional  FORECAST  » TMM 

(Although  there  are  hard  data  concerning  LTTA,  the  subjects  used  had  no  experience  but  had  received  long 
initial  formal  training.) 

“Head/Book”  Trade-Off  Analysis 

Currently,  there  are  no  hard  data  which  indicate  the  relative  effectiveness  between  FPJPA  and 
FORECAST  aids  for  reducing  entry  training  requirements.  However,  analyses  of  each  technology  or 
concept,  based  on  consideration  of  the  developmental  requirements  (including  task  identification  and 
analyses  (Tl&A)  products,  as  well  as  on  content  and  format  requirements,  indicate  that  the  FPJPA 
technology  requires  the  user  to  store  and  recall  less  guidance  and  information  for  task  performance  than  the 
other  technologies  or  concepts-thus  reducing  the  task  training  requirements  for  entry,  as  well  as  for 
experienced,  personnel.  Expressed  in  other  terms,  the  FPJPA  technology  relegates  more  maintenance 
guidance  and  information  to  the  “book”  and  less  to  the  “head”  than  do  other  technologies  or  concepts. 
Compared  on  the  basis  of  like  analyses,  the  comparative  potential  of  the  applicable  technologies  for 
reducing  entry  training  time  and  for  increasing  the  maintenance  efficiency  of  first  enlistment  personnel  is 
indicated  as  follows: 

1.  For  non-TS  tasks: 

a.  FPJPA  » New  Look 

2.  For  TS  tasks: 

a.  FPJPA  » Enriched  FORECAST  Enriched  LTTA 

(Enriched  rather  than  traditional  FORECAST  aids  are  included  in  these  statements.  Enriched  FORECAST 
aids  relegate  more  to  the  “book”  than  do  the  traditional  variety.) 

Based  on  the  results  of  these  analyses.  Table  4 indicates  combinations  of  technologies  which  would  be 
effective  for  reduction  of  initial  training  time  for  first  enlistment  personnel.  These  are  listed  in  the  order  of 
their  potential  for  reducing  initial  training  time  as  well  as  for  increasing  maintenance  efficiency. 


Tabic  4.  Effectiveness  of  Three  Combinations  of  IMG&I 
Technologies  for  Reducing  Initial  Training  Time 


Combination 

Non-TS 

Tasks 

TS  Tasks 

Order  of 
Effectiveness 

i 

FPJPA 

FPJPA 

i 

2 

FPJPA 

Enriched  FORECAST 

2.5 

3 

FPJPA 

Enriched  LTTA 

2.5 

The  indicated  relative  effectiveness  for  these  combinations  can  be  expressed  as  a formula; i.e„ 

1 » 2 % 3 

Enriched  LTTA  is  defined  as  a TS  aid  of  the  same  degree  of  enrichment  as  those  produced  for  the 
C-141  aircraft,  which  were  used  for  organizational  level  TS  in  the  Air  Force  Human  Resources  Laboratory 
(AFHRL)  study  reported  by  Potter  and  Thomas  (1976),  and  Potter,  Hubbert,  Landolfi,  Rice,  and  Kearns 


'V  Of  e quivalent  effectiveness. 


( 1976a.  1976b).  At  the  request  of  the  Air  Force  Logistics  Command  (AFLC),  AF'llKl  is  now  developing  a 
draft  specification  which  should  insure  the  development  of  LTTA  of  consistently  high  quality  from 
hardware  to  hardware.  Such  consistency  will  make  the  indicated  training  trade-off  for  a cluster  of  hardware 
possible.  The  current  specifications  for  developing  LTTA  do  not  ensure  such  consistency.  "New  Look"  aids 
lor  non-TS  tasks  are  not  included  in  Table  4 because  the  current  specifications  for  developing  these  aids 
have  a similar  consistency  weakness.  In  addition,  there  are  no  hard  data  available  that  attest  to  their 
effectiveness  for  supporting  the  performance  of  such  maintenance  tasks. 

Integrated  Implementation  of  FPJPA 
and  TOT  Technologies 

Figure  1 provides  a summary  of  results  of  key  research  efforts  concerning  the  effectiveness  of  TOT 
technology.  Six  of  the  1 1 TOT  efforts  indicated  were  supported  by  the  Army,  three  by  the  Air  Force,  and 
two  by  the  Navy.  Most  of  these  1 1 efforts  concern  training  for  maintenance  jobs  supported  by  the  TMM. 
Since  the  TMM  provides  much  less  guidance  and  in  most  cases  less  information  than  any  of  the 
combinations  shown  in  Table  4,  the  maintainer  must  compensate  for  the  TMM  deficiencies.  Of  course,  in 
this  case  a relatively  longer  and,  therefore,  more  costly  TOT  program  is  requied  than  when  more  effective 
maintenance  guidance  and  information  carrry  most  of  the  guidance  load. 

This  “head/book"  trade-off  problem  has  been  addressed  by  a few  reserachers.  Tire  Navy  report  by 
Theisen,  Elliott  and  Fishburne  (1978),  which  is  summarized  in  Appendix  H,  and  the  Air  Force  report  by 
Mullen  and  Joyce  (1974)  concern  integrated  applications  of  FPJPA  and  TOT  technologies,  and  the  report 
by  Shriver,  Fink,  and  Trexler  (1964)  concerns  the  integrated  application  of  FORECAST  and  TOT 
technologies.  Both  integrations  have  proven  effective;  however,  the  FPJPA/TOT  integration  will  usually 
result  in  shorter  training  time.  Since  the  thrust  of  this  report  is  the  maximization  of  on-the-job  time  for  first 
enlistment  maintenance  personnel,  the  discussions  that  follow  will  consider  the  potential  impact  of 
FPJPA/TOT  integrations  (Combination  1 shown  in  Table  4)  on  DOD  personnel  and  training  systems.  But 
most  of  what  is  said  would  also  apply  to  an  integration  of  FPJPA-FORECAST  Combination  2 (Table  4)  and 
TOT.  However,  the  TOT  would  be  appreciably  longer  because  less  of  the  guidance  and  information  in 
FORECAST  has  been  relegated  to  the  book. 

Considering  the  transitory  nature  of  maintenance  personnel,  as  well  as  their  training,  logic  would 
dictate  that  maximum  utilization  be  made  of  the  “book”  when  determining  the  head/book  trade-off.  What 
is  put  in  the  “book”  remains  in  the  book,  the  quality  and  quantity  of  its  contents  do  not  deteriorate  with 
time,  all  copies  of  the  book  are  identical,  and  the  content  of  the  book  will  never  deteriorate  from  lack  of 
use.  On  the  other  hand,  what  is  relegated  to  the  “head,”  or  training,  may  never  be  fully  and  accurately 
assimilated  by  many  individual  maintained,  and  the  amount  of  assimilation  will  vary  from  individual  to 
individual.  Once  an  individual  has  attained  the  desired  task  proficiency,  such  proficiency  will  deteriorate 
rather  rapidly  with  lack  of  use.  In  addition,  the  individual’s  stay  in  DOD  in  most  cases  is  relatively  short,  so 
whatever  proficiency  he  has  is  soon  lost,  and  the  training  cycle  must  be  repeated  for  his  replacement,  ad 
infinitum. 

The  current  state-of-the-art  for  IMG&I/TOT  reflected  by  Theisen  et  al.  (1978),  Mullen  and  Joyce 
( 1974),  and  Shriver  et  al.  (1964)  attests  to  the  feasibility  of  such  trade-offs.  In  addition,  the  AFHRL  draft 
FPJPA  spe  .ification  and  the  handbooks  provide  some  guidance  for  FPJPA/TOT  trade-off.  Substantial  DOD 
maintenance  gains  can  be  realized  by  the  implementation  of  the  current  statc-of-thc-art.  (However  the 
ground  rules  for  such  trade-offs  could  probably  be  greatly  refined  by  further  R&D.  The  accomplishment  of 
this  important  area  of  R&D  has  been  delayed  for  several  years  by  the  failure  of  appropriate  DOD  training 
establishments  to  support  it.  Hopefully,  one  outcome  of  the  current  R&D  program  of  the  Navy  Personnel 
Research  and  Development  Center  (NPRDC)  will  be  a refinement  of  IMG&I/TOT  trade-off  ground  rules.) 

Use  of  Test  Equipment.  An  often  neglected  key  factor,  concerning  head/book  trade-off  and  the 
pertormance  of  maintenance  tasks,  is  the  requirement  for  high  proficiency  in  use  of  test  equipment.  To 
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The  purpose  of  this  figure  is  to  identify  the  types  of  controlled  studies  supporting  IMG&I 
and/or  TOT,  to  give  the  general  flavor  of  their  results  and  to  direct  the  interested  reader  to  available 
summaries  and  lists  of  references  on  job  oriented  training,  technical  data,  and  performance 
measurement. 

The  armed  services  have  conducted  at  least  1 1 controlled  studies  concerning  the  effectiveness  of 
job  oriented  training  programs  for  maintenance.  The  experimental  job  oriented  training  programs 
developed  for  these  studies  have  been  aimed  at  several  types  of  job  situations.  Some  have  been 
structured  to  train  new  military  enlisted  personnel  to  perform  maintennace  jobs  involving  a number 
of  hardware  systems.  Other  programs  were  aimed  at  existing  single  system  jobs.  For  both  these  types 
or  programs,  the  existing  job  situations  were  accepted  without  change.  For  at  least  two  studies,  the 
job  situations  were  simplified  for  the  experimental  subject  by  modified  information  or  directions  for 
performing  job  tasks  and  the  TOT  program  for  each  was  developed  with  the  objective  of  training  the 
subjects  to  perform  the  tasks  of  the  simplified  job. 

The  effectiveness  of  each  of  these  1 1 experimental  job  oriented  training  programs  was 
compared  under  controlled  conditions  with  the  existing  theory  centered  training  programs  used  to 
train  similar  personnel.  A summary  of  most  of  these  studies  together  with  a list  of  reports  was  made 
by  Foley  (1967).  Further  information  concerning  the  more  recent  studies  can  be  obtained  from 
Elliott  (1967a),  Elliott  and  Joyce  (1968),  Chalupsky  and  Kopf  (1967),  Pieper  et  al.,  (1970),  Pickering 
and  Anderson  (1966),  Steinemann  and  Hooprich  (1967),  Steinemann  et  al.  (1967),  Van  Matre  and 
Steineman  (1966),  Mullen  and  Joyce  (1974),  Shriver  et  al.  (1964)  and  Theisen  et  al.  (1978). 

The  last  two  documents  report  hard  data  which  attest  to  the  feasibility  of  and  training  gains 
from  IMG&I/TOT  trade-offs.  The  report  by  Shriver  et  al.,  pertains  to  FORECAST  TS  aids/TOT 
trade-offs  applied  to  a small  cluster  of  electronic  hardware,  whereas  the  report  by  Theisen  et  al. 
pertains  to  a FPJPA/TOT  trade-off  for  one  hardware  item.  The  Mullen  and  Joyce  report  describes  the 
demonstration  of  another  FPJPA/TOT  trade-off  but  contains  no  comparative  hard  data. 

An  indepth  review  of  all  these  studies  indicates  that  (a)  job  oriented  training  subjects  could 
perform  the  tasks  of  their  jobs  with  much  less  on-the-job  training  (OJT)  than  the  control  subjects,  (b) 
medium  and  some  low  aptitude  subjects  were  more  successful  in  job  oriented  training  programs  than 
in  traditional  training  programs,  (c)  job  oriented  training  courses  were  shorter  than  conventional 
courses,  and  (d)  when  the  jobs  were  modified  with  improved  job  instructions  and  information,  the 
TOT  was  much  shorter. 

An  indirect  source  of  insight  as  to  the  questionable  effectiveness  of  theory-based  maintenance 
training  programs  is  found  in  research  on  performance  measurement.  The  student  measurement 
programs  in  all  theory-based  training  programs  are  heavily  weighted  with  paper-and-pencil  tests  of 
theory  and  job  knowledge.  A student  usually  learns  well  those  things  on  which  he  is  tested.  Foley's 
summary  (1974)  of  the  available  performance  measurement  research  literature  indicates  that  the 
relationship  between  the  test  subjects'  ability  to  actually  perform  the  tasks  of  maintenance  jobs  and 
their  scores  obtained  on  theory  or  job  knowledge  tests  is  very  low.  Likewise,  the  correlation  is  low 
between  school  marks  and  measures  of  job  performance.  A strong  inference  can  be  draw  from  these. 
The  only  way  most  personnel  learn  to  perform  job  tasks  is  by  practicing  those  job  tasks  properly. 
Learning  job  tasks  by  practice  is  what  TOT  is  all  about. 

Figure  1.  Research  efforts  supporting  IMG&I/TOT  trade-off. 

date,  most  IMG&I  developments  have  assigned  the  use  of  all  test  equipment  functions  to  the  “head”  rather 
than  the  book.  Under  such  conditions,  the  TOT  must  contain  sufficient  practice  of  test  equipment 
functions  to  ensure  “over  learning.”  But,  this  is  not  enough,  there  must  be  periodic  performance  tests  in  the 
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field  to  ensure  that  maintenance  personnel  sustain  this  necessary  high  proficiency.  The  model  JTPT  battery 
reported  by  Shriver  and  Foley  (1974a)  provides  a sub-battery  of  JTPT  for  test  equipment.  Model  task 
oriented  programmed  instruction  packages  (i.e.,  task  oriented  training)  are  available  which  provide  “hands 
on”  practice  in  the  use  of  general  electronic  test  equipment  (see  Scott  <x  Joyce,  1975a  through  19751). 
Since  such  general  test  equipment  are  used  across  many  hardware  clusters,  a most  cost  effective  action  for 
improving  the  quality  of  maintenance  even  with  TMM  would  he  a DOD-wide  program  for  improving  test 
equipment  proficiency.  Such  a program  would  require  only  modifications  and  expansions  of  the  available 
model  TOT  packages  and  JTPT  battery. 

Maximum  Job  Utilization  of  First  Enlistment  Personnel 

Attention  is  directed  to  the  portion  of  Table  5 concerning  entry  into  the  first  enlistment.  This 
portion  of  the  table  summarizes  both  the  training  gains  and  maintenance  gains  which  can  be  expected  from 
a quality  implementation  of  the  FPJPA  and  TOT  technologies.  The  gains  would,  without  a doubt,  result  in 
tremendous  reductions  in  LCC  of  hardware  ownership. 

Requirement  for  High  Quality  Backup  Support 

It  is  estimated,  based  on  hard  data,  that  entry  first  enlistment  personnel  having  received  high  quality 
TOT  can  accurately  isolate  90  to  95  percent  of  hardware  troubles  using  FPJPA  (see  Table  5,  column  5,  item 
3,  for  first  enlistment  personnel).  It  should  be  emphasized  that,  based  on  available  information,  this  level  of 
accuracy  greatly  exceeds  the  present  performance  of  most  DOD  maintainors.  This  superior  TS  performance 
alone  would  reduce  spare  parts  usage  and  secondary  damage  of  hardware  by  reducing  the  current  great 
reliance  on  excess  remove  and  replace  actions  for  fault  correction. 

But  even  though  a quality  integrated  implementation  of  FPJPA  and  TOT  technologies  would  greatly 
improve  the  current  efficiency  of  maintenance,  appropriate  steps  should  be  taken  to  insure  the  efficient 
solution  of  the  5 to  10  percent  of  faults  that  might  elude  isolation  by  FPJPA.  The  lower  portion  of  Table  5 
concerns  the  development  of  a cadre  of  backup  personnel  with  the  ability  to  efficiently  isolate  such  elusive 
faults,  as  well  as  to  perform  other  functions  which  will  be  discussed  later. 


111.  VERSATILE  MODEL  FOR  MAXIMIZING  BENEFITS  OF  FPJPA  AND  TOT  TECHNOLOGIES 

Figure  2 appears  as  the  foldout  at  the  end  of  this  report.  The  flow  chart  displayed  in  this  illustration 
represents  a modification  of  a chart  developed  by  Dr.  Robert  Blanchard  as  part  of  his  planning  for  the 
current  research  program  of  the  NPRDC,  San  Diego,  California  (Blanchard  & Laabs,  1978).  The  illustration 
also  contains  information  displayed  in  Table  5.  Most  of  the  information  found  in  that  table  is  inserted  at 
the  appropriate  places  in  the  flow  chart.  Much  of  the  chart  has  been  designed  to  be  self-explanatory,  but 
some  of  its  key  blocks  require  discussion.  To  facilitate  the  use  of  the  chart  in  the  discussions  that  follow, 
each  functional  block  on  the  chart  has  been  numbered. 

The  chart  (Figure  2)  is  divided  into  two  parts  by  a heavy  dashed  line  between  Blocks  1 1 and  12.  The 
portion  of  the  chart  to  the  left  of  this  dashed  line  is  designated  as  “First  Term”  functions.  As  used  in  this 
chart,  “First  Term”  is  applied  to  personnel  who  are  in  their  first  enlistment,  and  who  have  not  extended 
their  enlistments  to  take  advantage  of  the  broadbased  training  represented  by  Block  12.  The  portion  of  the 
chart  to  the  right  of  this  dashed  line  contains  “Second  Term”  functions.  “Second  Term”  is  applied  to  first 
enlistment  personnel  who  have  extended  their  enlistment  to  take  the  broadbased  training. 

In  many  respects  this  is  an  extremely  versatile  model.  For  example,  it  can  be  applied  to  one  hardware 
system  or  a cluster  of  hardware.  However,  to  obtain  the  maintenance  and  training  gains  indicated,  there  are 
five  extremely  firm  requirements: 

1.  Quality  FPJPAs  must  be  developed  for  all  the  hardware  in  the  cluster  under  consideration.  Such 
aids  must  reflect  the  key  characteristics  and  requirements  of  the  FPJPA  technology  as  discussed  in 
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Appendix  B.  These  include  Tl&A:  standard  language  and  keyed  pictorials  for  maintenance  instructions; 
appropriate  format  for  the  maintenance  environment;  as  well  as  ", hands  on  " validation  of  the  effectiveness 
of  the  product  by  the  contractor;  “ hands  on”  verification  of  the  effectiveness  of  the  product  by  the 
purchasing  DOD  agency;  and  warranty  of  the  product  by  the  contractor. 

2.  To  be  effective,  these  quality  FPJPAs  must  be  used  by  each  maintainer  while  he  is  performing  his 
job  tasks. 

3.  Quality  TOT  programs  must  be  developed  which  support  and  complement  the  FPJPA  for  the 
cluster  of  hardware  under  consideration.  The  fit  between  the  FPJPA  and  the  TOT  is  extremely  critical  for 
efficient  job  performance  of  first  term  personnel. 

4.  The  advancement  of  all  first  term  maintenance  personnel  both  in  skill  and  pay  levels  must  he 
accomplished  on  the  basis  of  demonstrated  ability  to  perform  job  tasks.  This  requires  the  elimination  of  all 
paper-and-pencil  theory  and  job  knowledge  tests  from  first  term  advancement  requirements.  (Scores  from 
such  paper-and-pencil  tests  have  little  relationship  to  ability  to  perform  maintenance  tasks  (Foley,  1974).) 

5.  After  initial  training,  each  first  term  maintainer  must  perform  both  non-TS  and  TS  maintenance 
tasks  on  a full  time  basis  for  at  least  one  year  using  FPJPA  (Block  6).  This  is  an  extremely  important 
requirement  for  the  cost  effective  utilization  of  first  enlistment  personnel.  Here  is  where  the  maintainer 
“pays  for  his  or  her  keep”;  demonstrates  his  potential  usefulness  to  DOD,  including  the  demonstration  of 
good  work  habits  and  of  ability  to  use  his  hands;  and  thus  earns  the  privilege  of  obtaining  advanced  training 
(Block  12). 

Key  Features  of  the  Model 

Current  Aptitude  Tests  (Block  I ).  Current  DOD  aptitude  tests  have  been  designed  to  select  people 
who  will  do  well  in  current  entry  maintenance  training  programs.  For  example,  many  current  Air  Force 
entry  maintenance  training  programs  require  people  with  aptitudes  of  80  percentile  and  above  to 
successfully  complete  current  academically  oriented  entry  training  programs.  A large  number  of  such 
personnel  do  not  aspire  to  make  maintenance  a career,  accounting  in  part  for  low  reenlistment  rates  as  well 
as  “early  outs.”  However,  both  TOT  and  FPJPA  research  have  indicated  that  most  middle  or  medium 
aptitude,  as  well  as  high  aptitude  personnel,  as  measured  by  current  aptitude  tests,  can  successfully 
complete  TOT  training  and  can  perform  maintenance  tasks  adequately.  So  for  the  present,  the  current 
aptitude  tests  can  be  used  for  selecting  entry  maintenance  personnel.  One  simply  has  to  adjust  the  aptitude 
cutoff  score. 

Short  Entry  TOT  (Block  3).  This  block  represents  integrated  applications  of  the  FPJPA  and  TOT 
technologies.  In  a short  summary  presentation;  such  as  this,  it  is  not  feasible  to  discuss  all  of  ramifications 
and  possibilities  for  such  integrated  applications,  but  a few  of  the  key  and  unique  characteristics  are 
mentioned. 

1 . The  meshing  of  the  content  of  the  FPJPA  for  the  cluster  of  hardware  under  consideration  and  the 
content  of  the  TOT  represented  by  this  block  is  an  absolutely  necessary  ingredient  for  such  applications. 
This  meshing  requires  precise  task  identification  for  all  hardware  selected  for  the  cluster,  followed  by  a very 
careful  trade-off  as  to  which  tasks  or  parts  of  tasks  for  each  hardware  will  be  relegated  to  TOT  and/or 
FPJPA.  To  achieve  this,  the  same  tradeoff  ground  rules  must  be  applied  for  each  hardware  in  the  cluster 
(see  Joyce,  Chenzoff  Mulligan,  <£  Mallory.  1973a,  1973b,  1973c). 

2.  TOT  packages  (for  initial  training)  must  be  developed  which  require  the  trainee  to  perform 
‘‘hands  on  ” practice  of  key  tasks  using  FPJPA  on  whichever  hardware  of  the  cluster  is  available  for  training 
purposes.  Since  the  trainee  will  receive  his  instructions  for  performing  such  “hands  on”  practice,  much  of 
the  TOT  package  will  be  a training  management  document  identifying  the  tasks  to  be  practiced.  Such  a 
TOT  package  will  also  provide  instructions  for  practice  on  those  activities  which  have  been  assigned  to 
training;  such  as  use  of  test  equipment.  Once  the  trainee  has  learned  to  perform  tasks'Wng  one  hardware  he 
will  be  able  to  perform  tasks  on  each  of  the  other  hardware  in  the  cluster  with  a minimum  of  cross-training. 
Such  cross-training  packages  must  be  developed  as  required,  covering  only  those  activities  not  included  in 
the  FPJPA  nor  in  the  initial  package. 
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3.  The  physical  location  of  the  entry  TOT  programs  can  cither  he  centralized  in  one  location  or 
decentralized:  i.e..  at  or  near  the  initial  job  assignment  of  each  trainee.  If  quality  FPJPA  and  quality  TOT 
packages  to  match  become  available,  on-site  job  entry  training  similar  to  that  utilized  by  the  Office 
Products  Division  of  IBM  (as  discussed  later  in  this  report)  may  become  the  most  cost  effective  route  for 
many  DOD  maintenance  situations,  fin  fact,  the  Army  is  now  planning  to  conduct  on-site  initial  training, 
hut  without  the  quality  FPJPA  discussed  here.) 

4.  The  proposed  TOT  would  practically  force  first  term  maintainers  to  use  FPJPA  to  perform  their 
tasks  because  that  will  be  the  only  way  these  people  will  have  been  trained  to  do  their  job  tasks.  This  would 
reduce  one  of  the  current  DOD  maintenance  problems,  that  of  getting  maintainers  to  use  their 
maintenance  instructions. 

Job  Task  Performance  Test  (JTPT)  Battery  (Block  4).  JTPT  battery,  similar  to  the  model  described 
by  Shriver  and  Foley  (1974a),  should  be  developed  for  each  cluster  of  hardware.  The  use  of  such  tests  is 
another  absolutely  necessary  ingredient  for  maximizing  the  utilization  of  first  term  maintainers.  The 
objective  of  the  proposed  TOT  programs  is  to  teach  people  to  perform  maintenance  tasks  of  a specific 
number  of  hardware  items  using  FPJPA-the  objective  is  not  to  teach  people  to  pass  papcr-and-pencil 
theory  and  job  knowledge  tests.  The  only  appropriate  tests  for  insuring  the  achievement  of  such  a task 
performance  objective  is  the  JTPT.  There  is  nothing  really  new  about  the  idea  of  using  the  JTPTs  as  training 
criteria.  They  were  used  extensively  by  the  Navy  and  Air  Force  during  and  after  World  War  II  (Foley, 
1977).  What  is  new,  is  their  refinement  (Shriver  & Foley,  1974a). 

In  addition  to  their  worth  for  ensuring  the  task  performance  ability  of  entry  personnel,  the  use  of 
JTPT  provides  the  basis  for  a great  improvement  of  the  DOD  posture  concerning  several 
equal-employment-opportunity  (F.F.O)  rulings  of  the  U.S.  Supreme  Court.  (These  rulings  have  indicated 
that  the  selection  of  employees,  as  well  as  their  promotion  should  be  based  on  ability  to  perform  job  tasks 
(Alluisi,  1977).)  It  is  worth  noting  that  the  Office  Products  Division  of  IBM  uses  task  performance  tests  as 
the  sole  criteria  for  successful  completion  of  entry  training  and  as  prime  criteria  for  all  career  advancement. 
(AFHRL  has  support  work  which  has  produced  some  graphic  symbolic  substitute  tests  of  promising 
empirical  validity  (Shriver  & Foley,  1974b).  However,  the  technology  for  developing  symbolics  requires 
more  refinement  before  such  tests  can  be  recommended  for  operational  use.) 

"Skill  Level”  Upgrade  on  First  Job  (Block  6).  The  award  of  a 4 skill  level  is  provided  after  about  6 
months  on  the  first  job.  This  skill  level  will  indicate  that  the  maintainer  has  demonstrated  his  ability  to 
perform  his  job  tasks  and  has  demonstrated  good  work  habits.  The  award  of  this  4 level  could  be  made  on 
the  basis  of  a supervisor's  evaluation.  This  is  a proposed  new  skill  level. 

Career  F. valuation  and  Selection  for  Broadbased  Training  (Blocks  7 through  12).  The  career 
evaluation  envisioned  for  Block  7 would  be  based  on  three  factors.  The  first  (a  go,  no-go  factor)  would  be  a 
comprehensive  JTPT  battery  which  would  ascertain  that  the  maintainer  could  perform  a wide  range  of 
maintenance  tasks  using  FPJPA  and  that  he  had  a high  degree  of  expertise  in  the  use  of  test  equipments. 
The  second  factor  would  be  a supervisor's  rating.  The  third  factor  would  be  an  aptitude  test  battery  for  the 
broadbased  training.  If  the  maintainer  successfully  completed  this  evaluation  process,  he  would  be  awarded 
the  5 skill  level  and  he  would  become  eligible  for  an  offer  to  extend  his  enlistment  for  broadbased  training 
(Block  12). 

Since  the  number  of  training  spaces  for  such  broadbased  training  would  be  limited,  a successful 
completion  of  the  career  evaluation  might  not  result  in  an  immediate  offer.  The  maintainer  might  have  to 
continue  working  in  the  Block  10  status  until  there  was  an  available  opening.  But  whenever  such  an  offer 
was  made,  it  would  be  accompanied  by  an  offer  of  a bonus.  The  amount  of  the  bonus  could  vary  with  the 
needs  of  the  services.  Any  5 level  individual  who  declined  to  extend  his  enlistment  (and  any  individual  who 
failed  the  Block  7 evaluation)  would  continue  to  work  in  a Block  10  status  as  indicated  on  the  flow  chart. 

Potential  Dollar  Gains  from  Improved  Utilization  of  First  Term  Personnel  (Blocks  6 and  10). 
Consider  those  personnel  that  remain  in  the  FPJPA  mode  for  their  entire  first  enlistment.  An  addition  of 
the  times  shown  in  Blocks  6 and  10  indicates  a total  of  from  3 to  3!4  years  of  effective  job  time.  Based  on 
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the  estimated  total  of  $60,000  personnel  cost  for  a 4-year  first  enlistment  individual,  mentioned  earlier;  3 
years  of  effective  job  time  would  cost  DOD  $20,000  per  year.  When  compared  to  some  individuals  in  the 
current  job  situation,  from  whom  only  one  year  of  effective  job  time  is  realized,  ($60,000  per  man-year), 
this  would  be  a saving  of  $40,000  per  man.  Applied  only  to  100  individuals,  this  would  result  in  a 
$4,000,000  saving.  Consider  also  the  individual  who,  after  18  months,  extends  his  enlistment  for 
broadbased  training  (Block  12).  Based  again  on  the  $60,000  figure  for  4 years.  DOD  will  have  received  1 
full  year  of  productive  labor  for  $22,500  before  he  extends.  And,  for  his  extended  4 years,  DOD  can  expect 
3 full  years  of  effective  job  time  at  $20,000  per  year.  These  manpower  savings  could  result  in  more 
qualified  maintenance  people  on  the  job  without  increasing  the  total  number  of  personnel  in  DOD.  And, 
these  savings  do  not  include  those  that  can  be  expected  from  improved  maintenance  efficiency.  The  types 
of  gains  being  considered  are  of  the  same  magnitude  as  those  realized  when  automobile  manufacturers 
changed  their  mode  of  operation  to  production  lines. 

Improved  Aptitude  Tests  (Block  8).  The  results  of  the  JTPT  batteries  administered  in  Blocks  4 and  7 
would  provide  a data  bank  for  the  development  of  an  improved  aptitude  test  battery.  Since  the  new 
aptitude  test  battery  would  be  standardized  against  measurements  of  ability  to  perform  job  tasks,  the 
objective  of  TOT,  the  prediction  of  such  a battery  for  the  initial  training  and  job  success  should  be 
improved  over  current  batteries.  The  use  of  aptitude  tests  based  on  ability  to  perform  job  tasks,  as  well  as, 
the  use  of  JTPT  for  determing  training  success  would  result  in  a great  improvement  of- the  DOD  posture 
concerning  the  previously  mentioned  EEO  rulings  of  the  U.S.  Supreme  Court.  The  Office  Products  Division 
of  IBM  has  developed  and  is  using  such  an  aptitude  test  battery,  which  has  been  validated  against  training 
success  and  field  performance,  as  measured  by  job  task  performance  (see  Appendix  K). 

Second  Term  Training  (Block  12).  It  is  envisioned  that  there  would  be  several  such  courses  in  each 
service,  each  course  being  aimed  at  an  extensive  cluster  of  hardware.  Each  such  hardware  cluster  would 
probably  reflect  a limited  number  of  vintages  of  hardware  design.  Selection  for  one  of  these  courses  would 
consider  both  the  needs  of  the  service  and  the  desires  of  the  individual.  An  individual  could  be  selected  for 
training  for  a cluster  of  hardware  which  does  not  match  his  initial  assignment.  The  personnel  selected  for 
the  broadbased  training,  indicated  in  Block  12,  would  have  several  advantages  over  those  now  receiving 
lengthy  entry  training.  An  important  advantage  is  a well  developed  real-world  frame  of  reference  concerning 
maintenance.  This  will  make  broadbased  training  much  more  effective.  Since  only  selected  experienced 
people  with  a long  term  commitment  would  attend  such  a course,  the  motivation  to  learn  should  be  high. 

Such  training  programs  to  be  effective  would  require  extremely  careful  planning  and  execution.  Each 
program  should  be  based  on  a careful  study  and  analysis  of  the  expected  job  utilization  of  its  graduates.  As 
indicated  in  the  task  guidance  display  for  Blocks  12  and  13,  graduates  would  use  FPJPA  for  most  tasks. 
They  would  also  be  expected  to  use  and  have  available  several  additional  categories  of  maintenance 
guidance  and  information  which  include  the  unique  SIMS  or  FOMM  categories;  i.e.,  maintenance 
dependency  charts  (MDC),  blocked  schematics,  and  keyed  information.  As  a result,  a large  portion  of  each 
broadbased  training  program  should  include  a large  amount  of  “hands  on”  activity  solving  many  difficult 
maintenance  problems  using  these  documents. 

Utilization  of  Graduates  of  Second  Term  Training  (Block  13).  The  graduate  of  one  of  these 
broadbased  courses  would  truly  be  a generalist  and  cculd  provide  several  valuable  functions.  They  could 
provide  backup  for  first  term  personnel  assigned  to  hardware  with  FPJPA.  They  would  also  be  expected  to 
perform  maintenance  on  hardware  for  which  no  FPJPA  is  available,  including  the  performance  of 
maintenance  on  transient  aircraft. 

A probably  more  effective  and  less  costly  scheme  for  utilizing  some  of  these  graduates  and  for 
providing  at  least  part  of  the  required  backup  capabilities  would  be  a centralized  maintenance  information 
center,  which  could  be  called  by  telephone  whenever  a difficult  problem  is  encountered.  A highly  trained 
expert,  for  each  hardware  system  being  maintained,  would  be  assigned  to  such  a center.  Such  a center  could 
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reduce  the  number  of  highly  trained  backup  personnel  required  in  the  field  for  all  hardware.  In  cases  where 
only  a few  items  of  a specific  hardware  have  been  procured  by  DOD,  it  could  possibly  be  used  as  a 
substitute  for  costly  maintenance  instructions  and  special  training.  This  concept  was  recommended  in  one 
of  the  early  AFHRL  job  performance  aid  technical  reports  (Folley  & Shettel,  1962).  A modification  of  this 
concept  is  being  used  successfully  by  the  Office  Products  Division  of  IBM. 

Appropriate  Utilization  of  SIMS  anf  FOMM  (Blocks  12  and  13).  An  appropriate  utilization  of  SIMS 
type  data  including  FOMM  is  also  indicated  on  the  flow  chart  (Figure  2)  in  conjunction  with  Blocks  1 2 and 
13.  As  mentioned  previously  hard  data  indicate  that  such  aids  improve  the  TS  performance  of  highly 
trained  and  experienced  personnel.  Three  versions  of  SIMS  data  are  indicated  on  Table  2;  i.e.,  traditional 
SIMS,  Air  Force  SIMS  and  FOMM.  The  hard  data  concerning  SIMS  was  gathered  from  the  use  of  Air  Force 
type  of  SIMS.  Currently,  there  are  no  data  which  indicate  that  FOMM  is  more  effective  than  Air  Force 
SIMS. 

There  are  only  three  categories  of  SIMS  type  aids  which  impact  TS  performance  more  than  TMM  and 
these  are  included  in  the  list  of  nine  guidance  and  information  categories  for  experienced  and  highly  trained 
personnel  (see  Figure  2,  Blocks  12  and  13).  They  are  the  MDC,  blocked  schematics,  and  keyed  information. 
Although  tire  specifications  cited  in  Table  2 for  traditional  SIMS  and  for  FOMM  provide  for  some 
step-by-step  instructions  for  non-TS  tasks,  these  instructions  are  of  the  traditional  variety  found  in  TMM. 
(FPJPA  for  non-TS  tasks  are,  therefore,  much  more  effective.) 

The  MDC  is  the  heart  of  the  SIMS  TS  concept;  however,  the  human  factors  characteristics  of  the 
MDC  vary  greatly  among  the  various  versions  of  SIMS.  An  analysis  of  the  characteristics  displayed  in  Table 
6 indicates  that  the  Air  Force  type  of  SIMS  is  superior  from  a human  factors  point  of  view,  containing 
some  of  the  features  of  the  FPJPA  technology.  The  Air  Force  MDCs  contains  step-by-step  checkout 
procedures  in  standard  language  and  each  component  mentioned  is  keyed  to  a locator  pictorial.  In  addition, 
the  symbology  used  for  indicating  dependencies  in  Air  Force  MDCs  is  the  least  complex  of  those  used  for 
these  aids. 

When  the  checkout  procedures  of  the  MDC  identify  an  out-of-tolerance  condition,  the  chart  provides 
a set  of  dependent  functional  units,  the  fault  to  be  identified  being  within  the  set.  That  is,  it  gets  the 
maintainer  in  the  right  ball  park  fast.  (To  be  effective  each  set  of  dependencies  must  be  complete.)  From 
this  point  on,  he  must  plot  his  own  course  of  action  making  use  of  the  hardware  descriptions  found  in  the 
block  schematics  and  keyed  information.  It  is  during  this  “free  wheeling”  activity  that  the  maintainer  can 
make  costly  errors  resulting  in  inefficient  TS. 


IV.  CURRENT  R&D  EFFORTS 

Currently,  there  are  only  two  active  R&D  programs  in  DOD  concerning  IMG&I. 

Navy  R&D  Program 

The  Navy  is  the  only  service  which  has  consistently  supported  R&D  concerning  FPJPA/TOT 
trade-off.  Starting  in  1972,  the  Human  Factors  Engineering  Division  of  the  Naval  Air  Development  Center 
(NADC)  supported  an  effort  that  resulted  in  a controlled  comparative  study  (Theisen  et  al.,  1978)  which 
produced  hard  data  attesting  to  the  feasibility  and  effectiveness  of  the  FPJPA/TOT  trade-off.  Currently,  the 
NPRDC  at  San  Diego,  California,  has  a new  3-year  advanced  development  project  which  is  aimed  not  only 
at  refining  the  IMG&l/TOT  trade-off,  but  more  importantly  at  solving  the  personnel  and  training 
implementation  problems.  This  project.  No.  Z0828PN  entitled  Job  Performance  Aid  Test  and  Evaluation, 
should  result  in  the  necessary  guidelines  for  optimum  FPJPA/TOT  trade-offs.  Some  contract 
efforts  should  be  completed  by  late  1979. 
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Current  Air  Force  R&D 

The  Air  Force  supported  the  initial  development  work  of  the  task  identification  and  analysis  (TI&A) 
technology  which  has  been  basic  to  the  more  recent  developments  of  job  effective  1MG&1  and  TOT  by  the 
Army  and  Navy  (Miller,  1953;  Miller  & Folley,  1951).  Also,  the  Air  Force  conducted  the  first  TOT  study 
which  started  in  1953  (Foley,  1964)  and  supported  most  of  the  exploratory  development  and  early 
advanced  development  of  the  FPJPA  technology  (see  Appendix  D).  In  addition,  the  Air  Force  conducted  a 
preliminary,  uncontrolled  demonstration  of  FPJP A/TOT  trade-off  (Mullen  & Joyce,  1974).  However,  the 
Air  Force  did  not  fund  planned  controlled  comparative  research  concerning  such  trade-offs. 

However,  (as  mentioned  previously)  the  AFHRL,  at  the  request  of  Air  Force  Logistics  Command 
(AFLC),  is  now  developing  a specification  under  contract  for  Tl&A.  In  addition,  a specification  for  LTTA 
is  being  developed  under  the  same  contract.  Some  consideration  is  also  being  given  to  expand  this  same  mix 
of  1MG&I  (JGM  for  non-TS  tasks  and  LTTA  for  TS  tasks)  to  the  organizational  maintenance  of  other 
hardware,  as  well  as  to  intermediate  maintenance.  Such  improved  applications  should  greatly  improve  the 
efficiency  of  maintenance  but  will  not  set  the  stage  for  the  maximum  utilization  of  first  enlistment 
personnel  by  effective  FPJPA/TOT  trade-off  as  indicated  in  Figure  2. 


V.  CURRENT  IMPLEMENTATION  ACTIVITIES 
Army  Implementation  Activities 

It  will  be  recalled  that  the  Army  supported  over  half  of  the  early  R&D  concerning  TOT,  including  the 
important  FORECAST  effort  which,  also,  produced  the  effective  FORECAST  TS  aids  discussed  previously. 
And  the  Army  is  now  in  the  forefront  concerning  implementation.  The  Army’s  Integrated  Technical 
Documentation  and  Training  (ITDT)  Program  is  by  far  the  boldest  and  most  far  reaching  implementation 
action  ever  taken  to  exploit  the  benefits  of  IMG&I,  TOT,  and  the  IMG&I/TOT  trade-off.  This  program 
already  contains  many  of  the  aspects,  displayed  in  Figure  2,  for  shortening  of  initial  training  of  first 
enlistment  personnel,  as  well  as  the  decentralization  of  such  training  by  providing  TOT  packages  and  by 
assigning  the  responsibility  for  initial  training  to  operational  units.  This  program  is  being  institutionalized 
by  the  MIL-M-630XX  series  of  specifications.  As  will  be  discussed  later,  these  specifications  require  major 
modifications  if  the  expected  and  necessary  results  of  IMG&l  and  TOT  technologies  are  to  be  achieved. 
However,  they  do  provide  the  overall  framework  for  more  effective  and  efficient  utilization  of  first 
enlistment  personnel  provided  die  necessary  specification  modifications  are  made  and  adequate 
enforcement  mechanisms  are  developed  and  used.  More  details  concerning  Army  implementation  activities 
are  found  in  Appendix  I. 

Navy  Work  Packages 

The  Navy  Air  Systems  Command  (NAVAIR)  has  developed  the  Mil-M-819XX(AS)  series  of 
specifications  which  include  provisions  for  work  packages  that  are  a decided  improvement,  when  compared 
with  the  TMM.  Each  such  work  package  is  designed  to  support  the  performance  of  one  non-TS  or  TS 
maintenance  task.  A common  aspect  of  each  work  package  is  a consolidated  presentation  of  available 
information  and  guidance  pertaining  to  a task;  materials,  which  are  normally  dispersed  throughout  the 
TMM.  Since  each  work  package  is  task  oriented,  the  FPJPA  technology  and  the  work  package  concept  are 
compatible.  Many  work  packages  reflect,  at  least,  the  format  aspects  of  the  FPJPA  technology.  However, 
the  specifications  contain  no  firm  requirements  for  applying  the  FPJPA  technology-including  TI&A. 
NAVAIR  requires  the  development  of  work  packages  for  all  new  systems.  The  work  package  concept 
certainly  is  an  improvement  over  the  TMM  and  provides  the  framework  for  a full  implementation  of  the 
results  of  the  recently  completed  NADC  FPJPA/TOT  trade-off  R&D(Theisen  et  al.,  1978). 

Air  Force  Implementation  of  FPJPA  Technology 

As  far  as  the  Air  Force  implementation  of  the  FPJPA  technology  is  concerned,  the  technology,  as 
developed,  is  based  on  Tl&A  and  is  applicable  to  both  non-TS  and  TS  tasks  for  both  the  organizational  (O) 
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ami  intermediate  (I)  levels  of  maintenance.  However,  the  implementations,  to  date,  have  concerned  only 
non-TS  tasks  for  the  organizational  maintenance  of  aircraft.  Job  Guide  Manuals  (JGM)  for  organizational 
maintenance  in  FPJPA  format  have  been  developed  for  the  C-141 , B-52,  KC-135,  F-106  (partial),  F-5E,  and 
the  A-10  aircraft.  They  are  being  developed  for  the  F-16  and  are  planned  for  the  Advanced  Medium  STOL 
Transport  (AMST)  as  well  as  for  various  other  subsystems  for  the  Aeronautical  Systems  Division  (ASD)  and 
Electronic  Systems  Division  (ESD).  Such  JGM  were  also  planned  for  the  B-l.  Of  these  JGM  developments, 
those  for  the  C-141  are  the  only  manuals  which  reflect  the  TI&A  aspects  of  the  FPJPA  technology.  More 
details  concerning  the  Air  Force  applications  are  found  in  Appendix  J. 

Industrial  Implementation  Efforts 

Five  industrial  organizations  have  been  identified  for  their  advanced  approaches  to  Tl&A,  IMG&l, 
training  and/or  personnel  utilization:  two  steel  corporations,  Amico  Corp.  and  U.  S.  Steel;  Cleveland 
Engine  Plant  of  Ford  Motor  Company;  Office  Products  Division  of  IBM;  and  Bell  System.  These  advanced 
industrial  approaches  are  very  similar  to  those  presented  and  recommended  for  DOD  implementation.  All 
of  these  applications  can  be  traced  to  human  factors  R&D  supported  by  the  military  services.  In  general, 
the  military  services  have  been  ahead  of  industry  in  human  factors  R&D  for  maintenance,  but  behind  some 
industries  in  applications. 

The  Office  Products  Division  of  IBM  has  an  impressive  integrated  system  of  personnel  utilization. 
This  integrated  system  has  many  of  the  characteristics  of  the  proposed  program,  outlined  in  Figure  2, 
including  effective  IMG&I/TOT  trade-offs;  all  training  success  and  job  advancement  are  based  primarily  on 
ability  to  perform  job  tasks  as  measured  by  the  JTPT;  personnel  selection  procedures  have  been  validated 
against  such  tests;  the  responsibility  for  maintenance  manual  development,  engineering  changes,  aptitude 
test  development,  formal  training  development  and  execution,  field  training  packages  development,  and 
performance  assessment  center  operation  is  under  one  directorate;  and  the  actual  selection,  hiring,  and 
initial  training  of  maintenance  personnel  have  been  decentralized  and  are  the  responsibility  of  the  line 
managers  in  the  field.  An  important  aspect  of  this  IBM  program  for  DOD  executives  and  managers  is  that 
the  program  is  operating  and  has  been  operating  successfully  for  several  years.  Appendix  K provides  a more 
complete  description  of  this  IBM  system  of  personnel  utilization. 

The  Bell  System  uses  TI&A  extensively  for  solving  operator  and  maintenance  personnel  and  training 
problems.  A reflection  of  the  strong  interest  in  TI&A  by  the  Bell  System  is  its  sponsorship  of  a “Conference 
on  Uses  of  Task  Analysis  in  the  Bell  System,”  in  October  1972.  However,  the  Bell  System  approach  does 
not  generally  include  formal  and  precise  trade-offs  of  TOT  and  IMG&I.  The  applications  made  by  Bell 
Systems  are  for  solving  specific  problems.  Important  ingredients  for  ongoing  applications  of  the  FPJPA 
technology  by  Bell  System  have  been  the  development  of  an  internal  guide  for  the  preparation  of  FPJPA 
and  an  accompanying  training  program  for  FPJPA  developers.  These  efforts  were  accomplished  under  the 
auspices  of  the  Bell  Laboratories.  With  some  modifications,  this  training  is  now  one  of  the  regular  and 
popular  courses  given  by  Bell  Laboratories  at  Piscatway,  New  Jersey,  for  managers,  the  idea  being  to  greatly 
reduce  or  eliminate  much  costly  training  by  using  good  job  instructions.  Another  version  of  this  training 
program  is  given  by  New  England  Bell  at  Marlborough,  Massachusetts  (see  Appendix  L for  more  detail). 

The  Armco  and  U.  S.  Steel  IMG&I/TOT  applications  are  very  similar,  both  being  originally  applied  to 
the  maintenance  of  the  electronic  controls  on  their  rolling  mills.  These  two  organizations  turned  to  the 
IMG&l/TOT  technologies  after  traditional  training  approaches  resulted  in  expensive  maintenance  failures. 
Both  organizations  have  now  applied  IMG&I  to  other  maintenance  problems.  A description  and  discussion 
concerning  the  Armco  experience  are  reported  in  an  issue  of  Iron  Age  (Snodgrass,  1976).  A summary  of 
this  Armco  experience  is  provided  in  Appendix  M.  More  recently,  the  Cleveland  Engine  Plant  of  the  Ford 
Motor  Company  is  having  an  IMG&I/TOT  program  developed  for  the  maintenance  of  its  programmed 
control  production  equipment. 
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VI.  PROBLEMS  AND  POLICIES  WHICH  IMPEDE  IMPLEMENTATION 


General 

1.  DOD  lacks  a department-wide  policy  requiring  that  only  those  technologies  and  concepts  whose 
effectiveness  is  supported  by  comparative  hard  data  should  be  considered  for  implementation.  Sales  talk 
and  “eye  appeal”  are  unacceptable  bases  for  expending  large  sums  of  operational  money  for  untested 
concepts  or  proposed  technologies  even  if  they  are  innovative.  The  only  concepts  and  technologies  that 
should  be  implemented  for  operational  purposes  are  those  whose  effectiveness  for  maintenance  and/or 
training  is  supported  by  comparative  hard  data  obtained  from  well  designed  and  controlled  tryouts. 

2.  Getting  well  developed  and  adequately  tested  technologies,  such  as  FPJPA  and  TOT. 
institutionalized  is  a perennial  problem,  especially  when  a technology  requires  fundamental  changes  in  long 
existing  programs,  procedures,  and  attitudes  of  entrenched  establishments.  The  Advanced  Systems  Division 
of  AFHRL  has  been  involved  in  the  implementation  of  several  well  developed  and  documented 
technologies,  such  as  FPJPA  and  instructional  systems  design  (1SD)  including  programmed  instruction  and 
job  (task)  oriented  training.  These  experiences  have  indicated  that  it  is  extremely  difficult  to  maintain  the 
integrity  of  a technology  during  its  so-called  implementation.  Operational  organizations  invariably  attempt 
to  implement  a much  “watered  down”  version  of  the  technology  and  consequently  obtain  much  “watered 
down”  results.  In  some  cases,  only  cosmetic  changes  to  existing  programs  are  reported  as  implementations. 
Currently,  it  requires  years  of  persistent  effort  on  the  part  of  the  research  community  to  get  a technology 
properly  institutionalized.  All  of  the  current  operational  implementations  of  the  FPJPA,  the  FORCAST, 
and  TOT  technologies  have  suffered  or  are  suffering  from  this  “watering  down”  phenomenon.  Examples 
include  most  Air  Force  developments  of  so-called  JGMs  for  organizational  maintenance  for  existing  and 
new  weapons  systems  (Mil-M-38800,  as  well  as  Mil-M-83495),  Army  Developments  of  Integrated  Technical 
Documentation  and  Training  (ITDT)  systems,  and  the  Air  Force  TOT  implementations,  including  BRIGHT 
SPARK  and  ABLE  CHIEF. 

3.  A mechanism  must  be  developed  for  the  timely  institutionalization  of  each  promising  new 
technology,  which  will  ensure  its  integrity.  A mechanism  for  the  orderly  implementation  of  technologies 
similar  to  that  used  for  new  weapons  systems  is  recommended.  Such  a mechanism  must  make  efficient  and 
effective  use  of  the  “know  how”  of  the  developers  of  the  technology  and  make  them  responsible  and 
accountable  for  its  implementation.  A new  technology  should  not  be  turned  over  to  a using  command  for 
its  operation  until  it  is  in  place,  “debugged”  and  operational  — just  as  a new  weapons  system  is  not  turned 
over  to  an  operational  command  until  it  has  been  “debugged”  and  proven  to  be  ready  fdr  operational  use. 

4.  Although  the  proper  implementation  of  quality  TOT  and  FPJPA  technologies  will  result  in 
significant  savings  in  overall  maintenance  labor  costs  and  in  substantial  reductions  in  spare  parts 
consumption,  their  implementation  will  require  relatively  large  investments  in  quality  maintenance 
guidance  and  information  and  in  quality  TOT  programs,  as  well  as,  changes  in  personnel  and  training 
systems.  Such  necessary  changes  will  require  an  integrated  and  concerted  effort  on  the  parts  of  the 
maintenance,  technical  data,  training,  personnel,  and  R&D  establishments.  All  actions  of  the  affected 
establishments  must  be  aimed  at  maximizing  the  utilization  and  efficiency  of  first  term  maintenance 
personnel. 

Personnel  Establishments 

1.  The  current  DOD  personnel  systems  for  maintenance  must  be  changed  to  better  reflect  and 
accommodate  the  various  vintages  of  hardware  and  the  great  proliferation  of  hardware  types  in  the  present 
and  future  DOD  inventories.  The  required  improvements  in  utilization  of  first  enlistment  personnel  will 
require  changes,  not  merely  cosmetic  patches  to  the  current  maintenance  personnel  systems.  However,  such 
changes  should  be  well  planned  and  implemented  in  manageable  increments  so  as  not  to  disrupt  the 
operations  of  the  services.  An  effective  mechanism  must  be  established  which  gives  primary  consideration 
to  the  needs,  not  necessarily  the  currently  expressed  wants  of  the  maintenance  systems.  The  mechanism 
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must  have  the  power  to  effect  the  necessary  changes.  In  this  regard  most  of  the  expressed  wants  of 
maintenance  managers  reflect  long  experience  with  traditional  training  and  maintenance  guidance.  They 
have  no  frames  of  reference  based  on  experience  with  1MG&1  and  TOT.  As  a result,  they  are  unable  to 
project  their  needs  in  terms  of  these  unfamiliar  technologies. 

2.  Large  portions  of  current  DOD  maintenance  personnel  systems  reflect  a policy  that  all  entry 
personnel  identified  for  maintenance  assignments  should  be  treated  as  if  most  or  all  such  personnel  were 
going  to  make  maintenance  of  military  hardware  a career.  This  policy  has  resulted  in  many  long  and 
expensive  career  oriented  training  programs  for  first  enlistment  personnel.  DOD  personnel  policy  should 
clearly  indicate  that  DOD  maintenance  personnel  should  not  be  considered  as,  or  treated  as,  career 
personnel  until  they  have  indicated  their  career  intentions  by  reenlisting  or  by  extending  their  first 
enlistments. 

3.  The  use  of  operational  functions  of  hardware  subsystems  as  the  primary  basis  for  maintenance 
assignment  codes  is  no  longer  a viable  concept.  In  addition,  the  personnel  system  must  be  changed  to  reflect 
the  fact  that  a major  portion  of  DOD  maintenance  tasks  must  be  performed  by  first  term  personnel.  What 
changes  should  be  made  will  depend  on  the  types  of  training  and/or  maintenance  guidance  documents  that 
are  to  be  used.  But  any  job  code  should  reflect  clusters  of  the  same  general  vintage  of  hardware,  as  well  as 
similar  training  and  maintenance  tasks  guidance. 

4.  Maintenance  personnel  systems  should  support  the  maximum  job  utilization  of  first  term 
personnel.  This  would  require  that  advancement  in  “skill  level”  and  pay  during  the  first  term  of 
maintenance  personnel  would  be  based  on  ability  to  perform  job  tasks.  (By  first  term  personnel  is  meant  all 
maintenance  personnel  in  their  first  enlistment  who  have  not  extended  their  enlistment  for  advanced 
broadbased  training  — see  Figure  2.)  All  paper-and-pencil  job-knowledge  and  theory  tests  should  be 
eliminated  for  these  first  term  maintenance  personnel. 

5.  To  implement  the  model  and  to  effect  the  savings  described  in  Figure  2 will  require  the 
reclustering  of  hardware  for  maintenance  assignment  codes,  for  first  term  maintainers  to  which  the  model  is 
applied.  This  will  probably  require  a new  maintenance  assignment  code  for  each  cluster  of  hardware  to 
which  the  FPJPA/TOT  trade-offs  are  applied.  Tire  number  of  hardware  in  a cluster  should  be  manageable. 
For  early  applications,  the  maintenance  of  each  cluster  should  require  a large  concentration  of  maintainers, 
and  if  possible,  all  hardware  in  the  cluster  should  reflect  a limited  number  of  design  vintages.  Such 
reclustering  is  required  because  all  FPJPA/TOT  trade-offs  must  be  based  on  the  same  trade-off  ground  rules, 
and  the  FPJPA  for  each  hardware  in  the  cluster  must  be  of  the  same  high  quality. 

6.  The  appropriate  application  of  TOT  and  FPJPA  would  make  the  clustering  of  hardware  systems  a 
less  difficult  task.  For  example,  all  of  the  electronic  subsystems  without  regard  to  their  function  for  one 
major  weapon  system,  or  for  several  similar  major  weapon  systems,  could  be  placed  in  one  assignment  code. 
Such  a clustering  would  be  possible  if  FPJPA  were  provided  for  all  hardware  in  each  cluster  and  if  all 
hardware  in  the  cluster  were  of  the  same  general  vintage. 

7.  Currently  many  maintenance  assignment  codes  not  only  cover  large  numbers  of  hardware 
subsystems  but  they  are  also  responsible  for  the  performance  of  both  non-TS  as  well  as  TS  tasks  at  both  O 
and  1 levels  of  maintenance.  The  difficulty  and  complexity  of  O level  tasks  seem  to  be  remaining  fairly 
constant  over  hardware  systems.  However,  I level  tasks,  especially  TS  tasks,  which  were  extremely  difficult 
in  the  past,  are  becoming  very  simple  on  some  new  hardware  systems.  Generally,  non-TS  tasks  are  less 
difficult  to  perform  than  TS  tasks.  In  clustering  tasks  for  maintenance  assignment  codes,  both  the  O/I 
maintenance  level  dichotomy  and  the  non-TS/TS  task  dichotomy  should  be  given  serious  consideration  as 
criteria  for  such  clustering. 

8.  In  further  regard  to  the  application  of  these  dichotomies,  the  type  of  job  performance  aids 
development  policy  must  be  reflected.  Currently  the  Air  Force  is  supplying  aids  with  some  characteristics 
of  the  FPJPA  technology  for  non-TS  tasks  at  the  O level  of  maintenance  of  new  weapons,  whereas  for  TS 
tasks  for  the  same  level,  either  symptom-cause  charts  or  LTTAs  with  uncontrolled  enrichment  being 
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developed.  For  both  non-TS  and  TS  tasks  at  the  I level  of  maintenance,  TMMs  are  currently  being 
produced.  Under  such  conditions,  a FPJPA/TOT  trade-off  would  be  only  possible  for  non-TS  tasks  at  the  0 
level.  (Of  course,  the  easiest  way  to  solve  this  problem  is  to  develop  FPJPA  for  all  tasks  at  both  levels  of 
maintenance.) 

9.  Since  the  development  of  improved  types  of  maintenance  guidance  is  both  the  Air  Force  and  the 
Army  is  being  obtained  for  new  weapons  systems,  clusters  of  tasks  for  each  new  weapons  system  should  be 
considered  as  the  prime  basis  for  new  maintenance  assignment  codes  for  such  systems.  If  overall 
cost-effectiveness  of  maintenance  is  the  objective,  tire  type  of  guidance  provided  for  all  tasks  included  in 
any  one  maintenance  assignment  code  must  be  of  the  same  type. 

10.  The  changes  in  maintenance  documents  and  training,  as  well  as  the  necessary  restructure  of 
personnel  systems  discussed  in  this  report,  would  cause  drastic  changes  in  the  content  of  many  military  and 
civilian  positions  in  the  maintenance,  training,  and  technical  data  establishments.  If  either  the  pay  or  the 
status  of  already-in-place  personnel  is  adversely  affected  by  such  changes,  the  necessary  changes  will  never 
be  truly  effected.  In  fact,  the  status  and  pay  of  those  people  expected  to  implement  the  necessary  changes 
should  be  enhanced.  The  DOD  personnel  establishments  should  follow  the  lead  of  a few  industrial 
establishments  who  protect  their  already-in-place  employees  from  the  adversities  resulting  from  change. 

Technical  Manual  Establishments 

1.  It  is  emphasized  that  the  realization  of  such  gains,  as  indicated  in  Figure  2,  requires  that  the 
integrity  of  advanced  technologies  be  maintained  during  their  implementation.  It  is  absolutely  necessary 
that  their  integrity  be  reflected  in  official  specifications,  guidance  documents,  policies,  and  well  written 
work  statements  in  contracts.  In  this  regard,  the  current  DOD  official  specifications  and  guidance 
concerning  IMG&I  technologies  have  deficiencies.  For  example,  Mil-M-38800  concerning  JGM  reflects 
format  aspects  of  the  FPJPA  technology,  but  not  the  important  TI&A  aspects.  Although  the  FPJPA 
technology  covers  both  non-TS  and  TS  tasks  for  both  I and  O levels  of  maintenance,  Mil-M-38800  is  limited 
to  only  non-TS  tasks  for  the  O level  maintenance  of  aircraft.  A more  recently  developed  Air  Force 
specification,  Mil-M-83495,  (although  limited  to  0 level)  covers  both  non-TS  and  TS  tasks  for  all  vehicles 
including  aircraft.  It,  too,  is  only  a format  specification.  It  calls  for  JGM  for  non-TS  tasks  which  greatly 
deviate  from  the  standard  language  and  locator  pictorial  requirements  of  the  FPJPA  technology.  For  TS 
tasks,  the  specification  calls  for  Fault  Isolation  Manuals  (FI)  which  are  modeled  after  an  Air  Transport 
Association  specification.  Currently  there  are  no  comparative  hard  data  available  which  would  support  the 
use  of  such  FI  in  DOD  maintenance  establishments.  However,  AFLC  has  requested  AFHRL  assistance  for 
correcting  at  least  part  of  these  shortcomings.  Mil-M-38799  concerning  Air  Force  type  SIMS  does  not 
require  TI&A.  The  Mil-M-630XX  series,  developed  by  Army  DARCOM,  provides  for  TI&A  but  not  for 
important  in-process  reviews.  In  addition,  the  effectiveness  of  the  types  of  IMG&I  specified  by  the 
Mil-M-630XX  series  documents  are  not  supported  by  comparative  hard  data. 

2.  To  develop  the  quality  of  maintenance  documents  necessary  to  effect  the  overall  maintenance 
improvements  and  reductions  in  maintenance  labor  costs,  will  require  at  least  from  50  to  100  percent  more 
money  than  the  Maintenance  Manual  Establishments  have  been  accustomed  to  spend  for  current 
maintenance  guidance  and  information.  The  budget  managers  must  realize  that  this  is  a one-time,  front-end 
investment  that  must  be  made  in  order  to  obtain  the  desired  system  lifetime  cost  reductions.  Whatever  is 
lacking  in  the  maintenance  guidance  documents  must  be  supplied  during  every  year  of  system  life  by  more 
training,  more  support  and  test  equipment,  more  spare  parts,  larger  inventories  of  mtqor  hardware 
components,  and  more  depot  maintenance. 

3.  For  maximum  first  enlistment  utilization,  based  on  current  research  findings,  aids  developed  in 
accordance  with  the  FPJPA  technology  are  required  for  both  non-TS  and  TS  tasks  for  both  O and  1 levels 
of  maintenance.  Such  aids  will  be  effective  for  both  high  and  average  aptitude  personnel. 

4.  To  be  effective  FORECAST  aids  for  TS,  developed  in  keeping  with  draft  Mil-M-632XX,  Part  1, 
will  require  more  initial  TOT  than  FPJPA  for  TS.  Such  FORECAST  aids  arc  probably  less  effective  for 


early  quality  performance  of  first-term  personnel  but  no  comparative  hard  data  are  available  to  support  this 
assumption.  No  personnel  aptitude  data  are  available  concerning  thses  aids. 

5.  LTTA  with  controlled  enrichment  such  as  those  used  in  the  recently  completed  AFHRL  advanced 
development  effort  (Potter  & Thomas,  1976)  will  improve  the  TS  performance  of  experienced  and 
conventionally  trained  personnel.  Such  aids  will  require  more  initial  TOT  than  FPJPA  for  TS  and  they  will 
probably  do  little  for  the  early  quality  performance  of  first  term  personnel.  The  effectiveness  of  such  aids 
has  only  been  tested  on  high  aptitude  personnel. 

6.  Maintenance  guidance  documents  with  uncontrolled  content  or  untested  results,  no  matter  how 
great  their  eye  appeal,  should  not  be  developed  or  purchases  for  use  by  first -enlistment  personnel.  Examples 
include  symptom-cause  charts  for  TS,  LTTA  with  uncontrolled  enrichment,  and  “new  look”  documents. 

7.  The  MDC  is  the  “heart”  of  the  SIMS  (including  FOMM)  concept  for  TS.  Hard  data  indicate  that 
experienced  and  highly  trained  personnel  can  find  more  equipment  faults  with  MDC  supported  by  keyed 
schematics  and  keyed  information.  Such  maintenance  guidance,  including  the  more  elaborate  FOMM,  is 
inappropriate  for  the  objective  of  maximizing  the  utilization  of  first-enlistment  personnel. 

8.  A much  higher  percent  of  hardware  faults  will  be  isolated  with  the  use  of  FPJPA  for  TS  or  LTTA 
(with  controlled  enrichment)  than  with  the  uce  of  traditional  data.  But  a backup  capability  of  highly 
trained  and  skilled  beyond-first -enlistment  personnel  will  be  required.  For  electronic  hardware,  quality 
schematics  and  theory  of  operation  will  be  required  for  these  people.  The  MDC  would  also  improve  their 
performance  if  the  added  expense  can  be  shown  to  be  cost  effective.  SIMS  and  FOMMs  both  can  provide 
quality  schematics  and  useful  theory  of  operation  and  should  be  considered  for  these  purposes. 

9.  A probably  more  effective  and  less  costly  scheme  for  providing  such  backup  capability  is  a 
centralized  maintenance  information  center,  which  can  be  called  by  phone  whenever  a difficult  problem  is 
encountered.  A highly  trained  expert  for  each  hardware  system  being  maintained  is  assigned  to  such  a 
center.  A modification  of  this  concept  is  being  used  successfully  by  the  Office  Products  Division  of  IBM. 

Training  Establishments 

1.  As  far  as  its  effectiveness  for  DOD  maintenance,  the  training  of  first  enlistment  personnel  is  a very 
costly  and  transitory  undertaking.  As  such,  its  length  should  be  minimized  and  decentralized  as  much  as 
possible.  The  IBM  use  of  TOT  programmed  instruction  packages,  coupled  with  improved  maintenance 
guidance  for  local  training  of  entry  personnel,  is  a model  that  should  be  given  serious  consideration  for 
DOD-wide  application.  If  such  a system  was  utilized  by  DOD,  both  the  TOT  training  packages  and  the 
improved  maintenance  guidance  would  have  to  be  tailored  for  novice  personnel.  But  TOT  and  FPJPA 
technologies  are  available  for  fulfilling  such  requirements  for  DOD.  As  indicated  previously,  the  Army  has 
taken  action  to  decentralize  its  entry  training  programs.  However,  few  Army  training  packages  and 
maintenance  guidance  documents  are  fully  developed  and  operational,  and  their  effectiveness  for  novice 
personnel  has  not  been  ascertained. 

2.  To  insure  maximum  cost  effectiveness  from  the  implementation  of  a FPJPA/TOT  model  similar 
to  that  shown  in  Figure  2,  the  training  and  job  success  of  first  term  DOD  maintainers  must  be  based  on 
demonstrated  ability  to  perform  job  tasks.  This  requires  the  utilization  of  JTPT  batteries  as  the  criteria  for 
training  and  job  success  of  first  term  maintainers,  as  well  as  the  abandonment  of  traditional 
paper-and-pencil  theory  and  job  knowledge  tests  for  such  personnel.  The  TOT  does  not  prepare  people  to 
pass  such  paper-and-pencil  tests.  With  regard  to  an  effective  JTPT  battery,  FPJPA  will  provide  for  much  of 
the  interface  between  maintainer  and  his  hardware.  Therefore,  if  the  maintainer  can  demonstrate  his  ability 
to  perform  a limited  number  of  key  and  difficult  tasks  of  his  job  (using  FPJPA),  plus  his  ability  to  use  test 
equipment  and  hand  tools,  it  can  be  assumed  for  sampling  purposes  that  he  can  perform  most  of  his  job 
tasks.  Because  of  this  characteristic  of  the  FPJPA/TOT  trade-off,  the  amount  and  cost  of  training  hardware 
necessary  to  provide  hands-on  practice  of  representative  tasks  for  quality  TOT  (based  on  a quality 
FPJPA/TOT  trade-off)  should  be  much  less  than  that  required  for  quality  TOT  supporting  the  TMM. 
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3.  It  is  proposed  that  the  responsibility  for  conducting  field  TOT  programs  be  given  the  maintenance 
establishment.  This  would  fix  the  responsibility  for  training  with  the  people  who  must  live  with  the  results. 
In  addition  to  conducting  the  broadbased  training  for  second  term  personnel,  the  training  establishments 
should  have  the  job  of  preparing  and  updating  the  TOT  packages,  as  well  as  of  training  the  appropriate 
members  of  the  maintenance  establishment  to  conduct  such  TOT. 

4.  Although  shorter  than  conventional  training,  quality  TOT  is  not  cheap  training.  Most  of  the 
training  time  (to  be  effective)  must  be  spent  in  “hands  on”  practice  of  key  job  tasks.  This  type  of  training 
cannot  be  obtained  in  “training”  facilities  consisting  of  classrooms,  podia,  tablet  annchairs,  blackboards, 
and  projectors.  Effective  TOT  requires  substantially  higher  instructor-to-student  ratios  than  DOD 
manpower  establishments  are  accustomed  to  tolerate.  Such  training  also  requires  larger  investments  in 
hardware,  hardware  simulators,  pictorial  simulators,  and  part-task  trainers.  So  the  per  week  or  per  day  cost 
of  TOT  will  be  greater  than  conventional  maintenance  training,  especially  for  electronic  hardware.  For 
example,  the  highly  concentrated  TOT  program,  which  is  only  a fourth  as  long  as  a conventional  program 
may  cost  four  or  five  times  as  much  on  a weekly  basis,  thus  making  the  total  operating  cost  of  producing  a 
graduate  the  same  or  slightly  more  than  the  total  cost  of  the  current  program.  Budget  managers  must  realize 
that  this  is  another  price  that  must  be  paid  to  reduce  the  LCC. 

5.  To  insure  the  integrity  and  effectiveness  of  both  the  initial  TOT  and  the  associated  JTPT 
batteries,  job  assessment  centers  are  proposed.  Each  proposed  center  would  operate  as  a joint  effort  with 
the  appropriate  training  establishment.  Selected  graduates  of  entry  training  programs  should  be  given  a 
battery  of  JTPT,  which  would  ascertain  the  effectiveness  of  entry  training  programs.  The  same  job 
assessment  center  could  be  used  to  ascertain  the  job  effectiveness  of  randomly  selected  experienced 
maintenance  personnel.  The  bank  of  data  obtained  from  such  centers  should  be  used  to  standardize  entry 
aptitude  tests. 

6.  As  to  the  effectiveness  and  future  of  centralized  and  general  training  for  first  term  maintenance 
personnel,  it  would  be  impossible  for  DOD  training  establishments  to  provide  “hands  on”  TOT  training  for 
the  maintenance  of  all  hardware  systems  and  subsystems  for  many  of  the  current  personnel  assignment 
codes.  In  this  regard,  the  present  long,  theory  bases  courses  now  given  to  entry  maintenance  personnel  for 
the  broadbased  assignment  (specialty)  codes  are  not  designed  to  permit  early  utilization  of  first  enlistment 
personnel.  The  only  hope  for  effective  and  efficient  first  enlistment  general  and/or  centralized  training  for 
maintenance  is  the  successful  application  of  the  following  hypothesis.  A formal  centralized  “hands  on” 
TOT  program  can  be  developed,  the  content  of  which  would  be  built  around  typical  hardware  from  a 
cluster  of  similar  hardware  and  would  be  ascertained  by  a very  careful  TOT/FPJPA  trade-off  for  that 
hardware.  During  this  course,  the  novice  would  learn  to  use  the  required  test  equipment  and  to  perform  all 
key  maintenance  tasks  on  the  selected  hardware.  His  success  would  be  determined  by  his  ability  to  pass  the 
JTPT.  FPJPA  based  on  similar  FPJPA/TOT  trade-offs  would  be  developed  for  the  other  hardware  of  the 
cluster.  A strong  hypothesis,  supported  by  some  research  results,  is  that  the  novice  would  be  able  to 
perform  maintenance  on  the  similar  hardware  using  FPJPA  with  a minimum  of  field  orientation  TOT  on 
the  unfamiliar  hardware.  The  “hands  on”  exercises  for  centralized  TOT  can  be  provided  on  a proper  mix  of 
actual  hardware,  part-task  trainers,  photographic  trainers,  and  computer  controlled  trainers.  The  testing  of 
this  hypothesis  is  one  of  the  R&D  efforts  which  the  Air  Force  training  establishment  has  not  yet 
undertaken. 

The  Maintenance  Establishment 

1.  If  improved  guidance  materials  for  the  performance  of  maintenance  tasks  are  to  be  effective,  the 
DOD  maintenance  establishments  must  properly  utilize  them.  This  will  require  changes  in  the  work  habits 
of  some  maintenance  personnel.  Such  changes  will  not  be  made  easily. 

2.  The  maintenance  establishments  must  also  make  maximum  use  of  first  enlistment  personnel.  Such 
personnel  will  be  assigned  to  many  maintenance  tasks  that  have  been  performed  in  the  past  by  people  with 
more  time  in  service.  This  will  require  changes  in  the  habits  and  attitudes  of  maintenance  supervisors  and 
superintendents. 
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3.  If  the  responsibility  for  conducting  initial  training  is  assigned  to  the  maintenance  establishments, 
they  must  be  manned  for  it.  Those  responsible  for  such  training  should  themselves  be  well  trained  for  the 
job.  Such  training  cannot  be  made  an  additional  duty  of  already  overworked  supervisors  who  are 
inadequately  prepared  for  such  tasks. 

4.  Such  changes  will  require  well  planned  and  executed  initial  implementations  and  long  term 
follow-ups.  Among  other  things,  supervisors  and  already-in-place  workers  must  learn  and  use  the  new 
maintenance  directions.  Not  liking  to  do  something  can  not  be  accepted  as  a valid  excuse  for  not  doing  it. 
Many  times  after  the  initial  resistance  to  change  is  overcome,  affected  people  wonder  why  they  were  against 
the  change.  For  example,  many  pilots  initially  resisted  the  requirement  for  them  to  conduct  preflight 
inspections,  but  now  it  is  an  unquestioned  part  of  a pilot’s  job. 

Integration  and  Implementation  Mechanisms 

1.  The  existence  of  specification  and  other  guidance  documents  (which  accurately  reflect  IMG&I 
and  TOT  technologies)  does  not  insure  that  individuals  within  the  affected  establishments  will  or  can 
properly  implement  such  technologies.  Technologies  (such  as,  FPJPA,  FORECAST,  TOT,  and  JTPT) 
require  many  people  in  the  training,  personnel,  maintenance,  maintenance  manual,  and  industrial 
establishments  to  modify  or  change  many  of  their  long-standing  attitudes  and  behaviors.  In  addition  to 
overcoming  their  natural  resistance  to  change,  many  of  these  people  must  learn  important  tasks  and  skills 
which  they  do  not  now  possess.  These  problems  call  for  well  planned  and  executed  actions  including  the 
following: 

a.  Appropriate  training  programs  concerning  improved  technologies  for  affected  managers, 
supervisors,  and  journeymen.  The  programs  should  result  in  certification  for  both  DOD  and  contractor 
personnel,  based  on  supervised  practice  and  demonstrated  ability  to  perform  the  required  skills.  These 
training  programs  should  stress  the  required  integration  of  technologies.  (The  certification  of  contractor 
personnel  may  require  some  modification  of  current  DOD  policies.) 

b.  The  establishment  of  DOD  in-house  capabilities  to  perform  in-process  review  of  contractor 
developed  subproducts  of  Tl&A  and  the  application  of  these  subproducts  to  1MG&1  and  TOT 
developments. 

c.  The  establishment  of  an  appropriate  mechanism  for  insuring  the  effective  integration  of 
IMG&I,  TOT,  and  JTPT  technologies. 

2.  The  tremendous  savings  to  be  realized  from  the  implementation  of  IMG&I,  TOT,  and  JTPT  will 
be  reflected  in  the  maintenance  establishments.  However,  the  substantial  investments  required  to  effect 
these  savings  must  be  made  by  the  technical  manual,  training,  and  personnel  establishments.  Currently,  the 
effectiveness  of  managers  in  each  of  these  affected  establishments  is  not  measured  by  the  savings  they 
effect  in  the  maintenance  establishments.  Current  funding  practice  uses  past  expenditures  for  each 
individual  establishment  as  budgeting  criteria  for  its  future  expenditures.  A mechanism  must  be  established 
which  will  provide  sufficient  additional  funds  for  these  necessary  investments  in  quality  IMG&I,  TOT,  and 
JTPT. 

3.  An  important  consideration  for  the  development  of  DOD  policy  encouraging  effective 
applications  of  IMG&I,  TOT,  and  JTPT  technologies  is  their  potential  for  rapid  buildup  of  both  new  and 
reserve  personnel  in  times  of  national  emergencies.  In-place,  quality  integrations  of  FPJPA,  TOT,  and  JTPT 
technologies  for  key  hardware  would  not  only  greatly  reduce  buildup  time  but  would  also  reduce  the 
number  of  spare  parts  used  at  a time  when  they  would  probably  be  in  short  supply. 

4.  Since  so  many  now  independently  operating  establishments  must  work  in  concert,  an  agency  must 
be  established  having  the  necessary  power  and  “know  how”  to  effect  the  necessary  changes.  The  Office 
Products  Division  of  IBM  accomplished  this  important  function  by  making  the  same  general  manager 
responsible  for  the  development  and  conduct  of  formal  training,  the  development  of  field  training  packages, 
the  development  of  on-the-job  task  guidance  documents,  the  development  and  validation  of  aptitude  tests. 
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and  the  conduct  of  (lie  job  assessment  center.  This  structure  provided  the  necessary  power  to  effect  such  an 
integration,  but  the  necessary  “know  how”  was  provided  by  an  expert  human  engineer,  and  management 
followed  his  directions.  The  application  of  power  unaccompanied  by  in-depth  “know  how”  is  a dangerous 
combination  when  dealing  with  complex  technologies. 

5.  The  Army  has  provided  the  controlling  function  for  their  Integrated  Technical  Documentation 
and  Training  (1TDT)  etforts  by  establishing  the  ITDT  General  Officer  Steering  Committee.  This  committee 
is  made  up  of  10  generals  and  one  civilian  director  holding  key  positions  from  the  (US  Army)  Training  and 
Doctrine  Command  (TRADOC),  Material  Development  and  Readiness  Command  (DARCOM)  and  Forces 
Command  (FORSCOM).  Such  a group  certainly  has  the  power  to  effect  such  an  integration.  For  success, 
ITDT  must  use  the  most  effective  maintenance  guidance  documents  that  the  state-of-the-art  can  produce; 
however,  the  specification  and  handbooks  provided  for  this  integration  will  not  ensure  the  development  of 
ettective  maintenance  guidance.  The  probable  consequence  will  be  lower  than  expected  results  and,  in  some 
cases,  failures,  which  should  not  be  blamed  on  weaknesses  of  either  the  FPJPA  or  TOT  technologies.  The 
ITDT  concept  is  the  first  large  scale  attempt  by  a service  of  DOD  to  effect  better  utilization  of  first 
enlistment  personnel  by  the  combined  use  of  TOT  and  improved  maintenance  instructions.  Such  a sincere 
and  worthy  eftort  can  tail,  however  if  the  effectiveness  of  the  maintenance  instructions  and  training  is  less 
than  the  state-of-the-art  can  provide. 
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APPENDIX  A:  TRADITIONAL  MAINTENANCE:  MANUALS  (TMM): 
A DLSCRIPTION  AND  MANPOWER  CHARACTERISTICS 


Traditional  maintenance  manuals  (called  technical  orders  (TO)  by  the  Air  Force)  have  come  to 
contain  three  types  of  data  that  directly  impact  maintenance;  namely,  (a)  step-by-step  directions  for  some 
non-troubleshooting  (non-TSO)  tasks,  (b)  limited  guidance  for  troubleshooting  (TS)  tasks  (such  as 
symptom-cause  charts  or  traditional  logic  trees),  and  (c)  hardware  information  including  schematics  and 
pictorials  to  support  TS. 

Traditional  Step-by-Step  Procedures  for  Non-TS  Tasks.  For  many  years  maintenance  manuals  have 
included  step-by-step  procedures  for  some  non-TS  tasks  such  as  checkout  procedures;  align,  adjust,  and 
calibrate  procedures;  remove,  replace,  and  repair  procedures;  and  servicing  procedures.  No  methods  or 
standards  were  available  to  evaluate  such  procedures  as  to  the  completeness  of  coverage  of  all  job  tasks,  as 
to  the  development  and  evaluation  of  such  directions  in  terms  of  a standard  language,  as  to  the 
development  and  evaluation  of  standard  pictorial  location  information  to  support  the  standard  directions, 
and  as  to  a determination  of  a consistent  level  of  detail  for  the  information  for  all  such  directions.  As  a 
result,  such  directions  are  not  written  in  standard  language,  directions  vary  in  length  and  complexity,  and 
very  few  pictorials  are  used  to  support  the  written  directions.  From  time  to  time,  ad  hoc  efforts  have  been 
made  by  various  individuals  and  groups  to  improve  such  instructions,  but  these  improvements  were  seldom 
if  ever  institutionalized.  Consequently,  improvements  have  not  had  wide  application. 

Traditional  Directions  for  TS.  Traditional  maintenance  manuals  provide  symptom-cause  charts  and/or 
unenriched  logic  trees  for  troubleshooting  guidance.  At  best,  either  of  these  items  provide  rather  limited 
coverage  to  the  problems  facing  a troubleshooter.  Symptom-cause  charts  do  not  contain  all  the  trouble 
symptoms  that  a system  can  develop,  and  many  times  the  number  of  causes  displayed  for  a given  symptom 
is  not  complete.  Logic  trees  in  their  traditional  form  do  not  display  appropriate  information  for  each 
decision  point.  When  using  either  of  these  directive  items,  the  technician  must  know  the  geography  of 
hardware  and  be  extremely  familiar  with  the  nomenclature  of  the  parts  of  the  specific  hardware  on  which 
he  is  working.  After  making  use  of  the  limited  guidance  provided  by  either  of  these  devices,  the  technician 
must  plot  his  own  course  of  action  and  glean  the  necessary  support  information  from  the  information 
categories  of  the  maintenance  manual.  To  successfully  use  these  materials  and  get  most  TS  tasks 
accomplished  in  a reasonable  time,  the  technician  must  be  highly  trained.  He  must  be  extremely  familiar 
with  the  TO  information,  with  the  geography  of  the  equipment,  and  with  the  function  of  each  unit  of  the 
equipment  under  test. 

Traditional  Information  Categories  for  TS.  In  addition  to  the  directions  found  in  symptom-cause 
charts  and  logic  trees,  a number  of  categories  of  information  which  support  TS  are  usually  presented  in 
traditional  maintenance  manuals.  These  categories  include  block  diagrams,  schematics,  flow  diagrams, 
normal  reading  and/or  tolerance  charts,  theory  of  operation,  pictorial  location  diagrams,  aircraft  wiring 
diagrams,  and  illustrated  parts  breakdowns.  Most  of  these  items  were  developed  to  support  the  original 
manufacture  of  the  hardware  and,  although  they  contain  much  information  of  use  to  the  troubleshooter, 
they  were  not  human  engineered  for  his  use  as  to  content,  format,  or  organization. 

The  isolation  or  identification  of  faulty  components  in  complex  mechanical  and/or  electronic  systems 
has  always  been  an  expensive  process  in  terms  of  technician  time  and  tire  number  of  spare  parts  used.  There 
is  some  evidence  which  indicates  that  many  technicians  have  only  a limited  facility  for  troubleshooting 
systematically.  When  this  limited  systematic  approach  is  not  successful,  they  fall  back  on  the  removal  and 
replacement  of  components  until  the  system  becomes  operational  (Folley  & Elliott,  1967).  Although  this 
shotgun  approach  usually  works,  it  is  very  expensive  in  terms  of  the  number  of  spare  parts  used,  and  the 
secondary  damage  caused  by  unproductive  and  unnecessary  remove  and  replace  actions. 
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Manpower  Characteristics  Required  for  Traditional 
Manuals  (Non-TS  Tasks  and  TS  Tasks) 

1.  The  maintainer  must  know  the  geography  of  the  hardware  on  which  he  is  working. 

2.  Tlte  human  must  make  up  for  the  weaknesses  or  lack  of  the  step-by-step  instructions  for  tasks. 
Where  instructions  do  not  exist  or  instructions  are  not  suffciently  comprehensive,  he  must  plot  his  own 
course  of  action.  Where  steps  or  cues  are  missing  in  task  instructions,  he  must  compensate  for  such 
deficiencies. 

3.  Many  maintenance  managers  contend  that  extensive  fundamentals  (theory)  training,  as  well  as 
knowledge  of  theory  of  operation  for  the  hardware,  are  necessary  prerequisites  for  performing  maintenance 
tasks  with  such  instructions.  These  contentions  are  especially  strong  in  electronic  maintenance 
establishments. 

4.  Only  high  aptitude  people  can  successfully  complete  such  training  programs. 

5.  There  is  no  doubt,  that  extensive  experience  in  maintaining  the  hardware  is  required  for  effective 
maintenance  with  such  uncontrolled  maintenance  guidance  and  information. 

6.  For  effective  maintenance,  the  maintainer  must  also  be  proficient  in  his  use  of  test  equipment. 

Added  Manpower  Characteristics  Required 
for  Traditional  Manuals  (TS  Tasks) 

All  of  the  characteristics  required  for  non-TS  tasks  also,  apply  to  TS  tasks.  In  addition,  TS  tasks  are 
generally  more  demanding  of  maintenance  technicians  than  non-TS  tasks.  There  are  many  factors  that 
contribute  to  this  demand,  some  of  which  follow. 

1.  The  troubleshooting  process  most  times  should  include  the  accomplishment  of  various  mures  of 
non-TS  tasks,  as  well  as  proficient  use  of  several  test  equipments. 

2.  Since  most  traditional  maintenance  manuals  contain  very  limited  directions  for  TS  tasks,  the 
maintenance  technicians  must  plot  their  own  course  through  this  maze  of  activities.  For  effective  TS,  using 
such  manuals,  the  technicians  must  possess  and  use  an  effective  TS  strategy. 

3.  Tlte  following  of  such  a strategy  usually  requires  a high  degree  of  persistence.  The  strategy  must 
be  interrupted  many  times  to  gather  test  point  information.  If  the  test  equipment  is  not  used  accurately, 
the  result  is  false  test  point  information  which  reduces  the  effectiveness  of  the  strategy.  There  is  always  the 
temptation  to  gather  more  than  the  necessary  information  in  the  area  of  a test  point  and,  thus,  lose  track  of 
the  strategy. 
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APPENDIX  B:  DESCRIPTION  OF  SEVERAL  TYPES  OF  IMPROVED 
MAINTENANCE  GUIDANCE  AND  INFORMATION  (IMG&I) 


During  the  past  20  years,  several  programs  have  been  directed  toward  improving  the  effectiveness  of 
maintenance  manuals  as  sources  of  job  directions  for  maintenance  at  the  Organizational  (0)  and 
Intermediate  (I)  levels.  Shriver  and  Trexler  (1966)  provide  a summary  of  most  of  the  earlier  efforts. 
However,  there  are  five  main  sources  of  new  categories  of  directions  and  information;  namely,  the  Symbolic 
Integrated  Maintenance  System  (SIMS)  concept,  the  Air  Force  Presentation  of  Information  for 
Maintenance  and  Operation  (PIMO)  Project,  the  work  of  the  Air  Force  Human  Resources  Laboratory 
(AFHRL),  the  Air  Force  Vietnamization  Projects,  and  the  Army’s  FORECAST  Project  accomplished  by  the 
Human  Resources  Research  Office  (HumRRO).  The  SIMS  concept  produced  a new  troubleshooting  (TS) 
tool,  i.e.,  the  maintenance  dependency  chart  (MDC),  as  well  as  reformatted  schematic  and  theory  of 
operation  information.  The  PIMO  project  produced  the  fully  proceduralized  job  performance  aid  (FPJPA) 
for  only  the  non-TS  tasks,  and  the  AFHRL  work  produced  FPJPA  for  both  non-TS  and  TS  tasks.  The 
PIMO  project,  among  other  things,  simplified  the  MDC.  The  Vietnamization  efforts  systematized  the 
production  of  FPJPA  for  non-TS  tasks  and  TS  tasks,  as  well  as  for  MDC.  The  FORECAST  project  was 
primarily  a training  project,  but  it  produced  another  type  of  TS  aid,  and  enriched  blocked  information  flow 
diagram  called  the  Cue  Response  Set. 

In  one  sense,  the  SIMS  effort  started  with  the  existing  technical  order  materials  and  attempted  to 
improve  format  and  content  of  TS  directions  and  information  for  use  by  the  maintenance  technician.  The 
AFHRL  work  started  with  a study  of  the  TS  and  non-TS  tasks  of  the  maintenance  job  and  developed  new 
types  of  directions  that  would  guide  the  performance  of  each  type  of  task.  The  PIMO  project  contains 
elements  of  both  these  approaches.  All  can  make  certain  contributions  toward  improving  the  efficiency  of 
maintenance  or  training  or  both.  An  overriding  consideration  for  the  application  of  any  of  these  concepts 
or  technologies  is  what  each  can  do  for  the  first  term  maintenance  technician  in  terms  of  increased  time 
during  this  enlistment  as  an  efficient  and  productive  worker.  A closely  related  factor  is  the  potential  of  each 
for  the  reduction  of  “buildup”  time  for  national  emergencies. 

Symbolic  Integrated  Maintenance  System  (SIMS)  Including  the  Functionally 
— Oriented  Maintenance  Manual  (FOMMs) 


The  original  system  provided  for  both  non-TS  and  TS  tasks;  the  directions  for  non-TS  tasks  generally 
had  the  same  limitations  as  traditional  maintenance  manuals.  The  main  thrust  of  SIMS  and  FOMMs  has 
concerned  improved  directions  and  information  for  TS.  For  TS,  the  SIMS  concept  has  substituted  a much 
more  powerful  aid  than  either  the  symptom-cause  chart  or  the  traditional  logic  tree:  the  MDC.  This  MDC  is 
in  the  form  of  a matrix.  It  is  a structured  presentation  of  both  directions  and  information.  On  the  Y axis  of 
this  matrix  is  placed  a systematic  step-by-step  checkout  procedure.  (Although  such  instructions  could  be 
presented  in  standard  direction  language,  currently  such  directions  vary  in  length  and  comprehensibility.) 
When  followed,  this  procedure  will  produce  out-of-tolerance  conditions  or  symptoms  when  the  system  is 
not  functioning  normally.  On  the  X-axis  of  this  matrix  are  placed  all  of  the  functional  units  of  the  system. 
A rather  complex  set  of  symbols  is  provided  for  the  cells  of  the  matrix  which  indicates  the  desired 
equipment  responses  for  each  step  of  the  checkout  procedure  and  identifies  the  set  of  functional  units  that 
can  cause  such  an  out  of  tolerance  condition.  But,  once  this  set  of  functional  units  (dependencies)  is 
identified,  the  technician  must  plot  his  own  course  for  determining  which  functional  unit  in  the  set  is  faulty 
and  to  troubleshoot  and  to  repair  the  faulty  functional  unit.  Great  care  must  be  taken  in  developing  MDC 
to  insure  that  each  set  of  functional  units  is  complete.  As  to  sets  of  symbols,  several  have  been  developed 
by  various  Navy  and  Air  Force  efforts.  The  degree  of  complexity  has  varied  greatly  from  set  to  set. 
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The  SIMS  concept  provided  improvements  in  organization  of  information  to  support  TS.  The  blocked 
schematic  of  the  SIMS  concept  combined  much  of  the  information  presented  in  traditional  maintenance 
manual  block  diagrams,  schematics,  flow  diagrams,  and  normal  reading  and/or  tolerance  charts.  The 
schematics  are  reorganized  and  placed  in  blocks  of  shades  of  gray  or  blue.  Each  block  (or  sub-block) 
represents  a black  box  or  a functional  unit  of  a black  box.  The  theory  of  operation  of  the  traditional 
maintenance  manuals  was  reorganized  into  similar  blocks  so  that  the  technician  can  more  easily  find  a 
statement  as  to  the  operational  function  represented  by  each  major  block  of  the  blocked  schematic.  In 
SIMS,  the  pictorial  location  diagrams  are  replaced  by  pictorials  with  location  grids  such  as  are  found  on 
road  maps.  Due  to  the  color  requirement,  the  printing  of  block  schematics  and  blocked  information  is 
expensive.  In  addition,  many  times  a block  is  not  large  enough  for  complete  functional  information. 

Navy  Implementations  of  SIMS  and  FOMMs.  Most  of  the  DOD  efforts  for  the  production  of  SIMS 
aids  have  been  accomplished  by  the  Navy.  Over  the  years  the  Navy  has  issued  three  specifications  based  on 
the  SIMS  concept  Mil-M-24100  (SHIPS),  I Dec  1964;  MU-M-24100A  (SHIPS),  15  Jun  1966,  and 
Mil-M-24100B,  2 Jan  1974.  The  first  two,  both  entitled  “Manuals,  Orders,  and  Other  Technical 
Instructions,  for  Equipment  and  Systems,”  follow  the  original  SIMS  concept  very  closely.  The  most  recent 
specification  (2  Jan  1974),  entitled  “Manuals,  Technical:  Functionally  Oriented  Maintenance  Manuals 
(FOMM)  for  Equipment  and  Systems”  calls  for  a more  complex  and  detailed  set  of  symbols  for  the  MDC. 
As  with  the  original  SIMS,  these  specifications  provide  for  a complete  data  package  for  both  TS  and  non-TS 
activities.  However,  the  directions  for  non-TS  activities  are  similar  to  those  provided  by  traditional 
maintenance  manuals. 

Air  Force  SIMS  Work 

The  Air  Force  operational  implementation  of  SIMS  type  aids  has  been  rather  meager  when  compared 
to  the  Navy.  Most  of  the  work  has  been  limited  to  tryouts  and  evaluations,  development  of  specifications 
and  very  limited  implementations.  All  Air  Force  SIMS  efforts  have  been  limited  to  the  support  of  TS  tasks. 

Air  Force  Logistics  Command  (AFLC)  supported  a rather  broadbased  development  and  tryout  of 
SIMS  type  aids  in  the  middle  60s.  The  aids  developed  for  this  project  included  traditional  SIMS 
information,  blocked  schematics,  and  blocked  information.  Although  a formal  report  of  the  outcome  was 
never  published,  the  results  of  tryouts  of  these  data  were  generally  positive.  One  output  of  this  tryout  was 
the  development  of  MDC,  blocked  schematics,  and  blocked  information  for  the  voice  omnirange 
equipment.  These  data  were  used  by  the  Air  Force  Communications  Service  (AFCS)  for  several  years.  When 
the  data  wore  out  from  use,  AFLC  was  unable  to  replace  them.  Another  output  of  this  effort  was  draft 
specification  Mil-M-38799  (USAF),  15  Oct  69. 

The  only  hard  data  available  which  compare  the  effectiveness  of  SIMS,  TS  aids  and  traditional 
maintenance  manuals  was  generated  by  the  PIMO  project.  This  project  evaluated  a modification  of  SIMS 
type  aids  for  organizational  troubleshooting  tasks.  The  aids  developed  for  the  C-141  aircraft  included  MDC, 
as  well  as  key  schematics  and  keyed  information.  The  results  indicated  that  experienced  maintenance 
technicians  could  find  90  percent  more  troubles  on  their  first  troubleshooting  attempts  than  when  using 
traditional  maintenance  manuals  (AFSC-TR-69- 155(1),  1969). 

The  requirement  for  hard  data  comparing  the  relative  effectiveness  of  FPJPA  and  SIMS  was  reflected 
in  the  original  AFHRL  advanced  development  plan  prepared  in  1967.  It  called  for  comparing  FPJPA,  SIMS 
and  traditional  maintenance  manuals.  In  anticipation  of  this  comparison,  SIMS  data  were  prepared  for  the 
Doppler  radar,  AN/APN-147,  and  for  its  computer,  AN/ASN-35.  Due  to  lack  of  funding,  the  comparison 
was  never  conducted.  As  a result,  the  requirement  for  such  comparative  hard  data  still  exists. 

The  only  official  Air  Force  SIMS  type  specification,  Mil-M-38799  (USAF),  was  published  15  Aug  71. 
This  document  called  for  an  improved  MDC.  Each  functional  unit  listed  on  the  X axis  is  keyed  to  a pictorial 
which  indicates  to  the  technician  how  the  functional  unit  looks  and  where  it  is  located.  All  of  the  hardware 
items  mentioned  in  the  checkout  procedures  are  also  keyed  to  pictorials.  Such  MDC  were  first  developed 
for  the  UH-1H  as  part  of  the  Vietnamization  program.  These  MDC  were  first  specified  in  Mil-J-83302,  the 
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Vietnamization  specification.  (It  should  be  noted  that  the  latest  FOMM  specification,-Mil-M-24100B  (2  Jan 
1974),  does  not  call  for  these  pictorial  locators.)  The  PI  MO  MIX'  and  the  MIX'  specified  in  Mil-J  -83302  and 
in  Mil-M-38799  (USAF)  (15  Aug  71)  called  for  a less  complex  set  of  symbols  than  that  required  by  the 
original  SIMS  concept  and  by  Mil-M-24100B. 

After  the  UH-1 11  effort,  the  job  performance  aids  produced  for  other  Vietnamization  projects  did  not 
include  SIMS  type  data.  Concerning  SIMS  type  data  for  Air  Force  hardware,  such  data  have  been  official 
Air  Force  technical  order  (TO)  materials  since  1968.  Their  status  has  been  that  they  were  available  to 
requesting  Air  Force  agencies  provided  that  their  requirement  was  justified  and  that  sufficient  funding  was 
available.  Their  development  has  not  been  mandatory.  No  Mil-M-38799  SIMS  type  data  have  been 
generated  for  the  Air  Force  since  their  official  acceptance. 

Evaluative  Statement 

SIMS  directions  (including  FOMMs)  for  non-TS  tasks  are  no  more  effective  than  similar  directions 
found  in  traditional  maintenance  manuals.  Well  designed  SIMS  aids  for  TS  tasks  will  improve  the 
troubleshooting  performance  of  many  highly  trained  and  experienced  maintenance  technicians.  All  users 
have  been  high  aptitude  personnel.  The  key  to  this  improvement  is  the  MDC  which  requires  the  user  to 
learn  a unique  and  rather  complex  symbology.  The  MDC  provides  a fixed  strategy  which  gets  the  user  into 
the  right  “ball  park.”  However,  SIMS  aids  will  do  little  to  increase  the  effectiveness  of  each  first  term 
maintenance  technician  in  terms  of  increased  time  as  an  efficient  and  productive  worker  or  to  reduce  the 
high  aptitude  requirement  of  such  personnel.  SIMS  type  data,  and  especially  FOMM,  are  very  expensive  to 
develop  and  produce.  In  addition,  to  date  no  one  has  given  adequate  consideration  to  the  problems  and 
costs  of  updating  such  data. 

FuDy  Proceduralized  Job  Performance  Aids  (FPJPA)  Technology 

FPJPA  is  the  generic  name  of  a technology  which  can  be  used  in  the  development  of  step-by-step 
directions  for  both  non-TS  and  TS  tasks.  The  current  technology  contains  aspects  obtained  from  several 
sources  (see  Table  1 in  this  report)  including  the  AFHRL  exploratory  development  efforts,  the  first 
AFHRL  advanced  development  effort  (1967-1972),  the  PIMO  project,  JPA  projects  for  Vietnamization 
and  the  development  of  aids  for  the  C-141  aircraft.  Apprendix  E provides  a summary  of  Air  Force,  Army 
and  Navy  R&D  FPJPA  efforts  concerning  only  non-TS  tasks.  In  addition,  the  Air  Force  and  Navy  have  had 
FPJPA  R&D  efforts  concerning  TS  tasks,  and  both  TS  and  non-TS  tasks.  The  Navy  has  completed  one 
effort  concerning  the  FPJPA/Task  Oriented  Training  (TOT)  trade-off  and  is  now  contracting  for  further 
R&D  concerning  this  area.  Each  of  the  efforts  will  be  discussed  at  an  appropriate  place  in  this  paper. 

The  FPJPA  concept  did  not  emerge  as  a true  technology  until  specification  and  guidance  documents 
were  developed  for  the  Vietnamization  projects  (Mil-J-83302).  Currently,  AFHRL-TR-7343,  a three 
volume  report,  provides  the  best  description  and  directions  concerning  the  FPJPA  technology  for  both 
non-TS  and  TS  tasks  (see  Joyce  et  al.,  1973a,  1973b,  1973c).  The  most  important  addition  to  this  report  is 
found  in  AFHRL-TR-75-38  (Shriver,  1975)  concerning  improved  cues  for  step-by-step  directions.  This 
AFHRL-TR-73-43  reflects  the  lessons  learned  in  applying  the  Vietnamization  documents.  To  assist  the 
reader  in  assessing  recent  maintenance  manual  developments,  several  key  characteristics  and  requirements 
of  the  FPJPA  technology  are  outlined  and  briefly  discussed. 

Task  Identification  and  Analyses  (TI&A)  of  Identified  Tasks 

A unique  characteristic  of  this  technology  is  the  formal  requirement  for  the  development  and  use  of 
TI&A.  To  ensure  the  effective  job  support  for  non-TS  by  FPJPA,  an  aid  must  be  considered  for  every  task 
of  a hardware  to  be  maintained.  Each  FPJPA  must  contain  accurate  and  complete  step-by-step  instructions, 
keyed  to  pictorial  locators  for  its  task.  And  each  FPJPA  must  contain  all  of  the  cues  necessary  for 
individuals  of  the  target  populations  to  perform  the  task. 


FPJPA  for  TS  tasks  must  be  built  around  a logic  or  troubleshooting  tree  that  identifies  all  the 
troubleshooting  signatures.  There  must  be  a signature  for  each  replaceable  component  assigned  to  the  level 
of  maintenance  for  which  the  TS  aid  is  being  developed.  For  each  decision  point  in  the  TS  tree,  accurate 
step-by-step  instructions,  with  the  necessary  cues,  must  be  developed. 

To  ensure  the  completeness  and  adequacy  of  the  required  product  of  formal  Tl&A,  there  must  be 
timely  periodic  in-process  reviews  and  official  acceptance  of  these  products  by  knowledgeable  DOD 
representatives.  These  products  then  become  the  criteria  for  the  structure  and  completeness  of  the  FPJPA 
to  be  used  during  their  development. 

Standard  Language  and  Keyed  Pictorials  for  Maintenance  Instructions 

Another  unique  characteristic  of  FPJPA  coming  from  the  PIMO  project  is  a standard  language.  Each 
maintenance  direction  is  limited  to  a total  of  three  sentences  and  a total  of  25  words.  Each  of  these 
sentences  is  limited  to  10  words.  The  first  word  of  each  sentence  is  an  action  verb  in  the  second  person 
taken  from  a standard  verb  list.  In  addition  to  standard  verbs,  all  nouns  are  cross-referenced  (keyed)  to  a 
locator  pictorial  which  indicates  to  the  user  the  appearance  of  the  item  and  its  location  in  the  hardware. 

This  standard  language  provision  coupled  with  the  pictorial  locators  reduce  tire  reading-level 
requirements  of  the  user  to  a minimum.  The  user  must  recognize  and  know  tire  meaning  of  a limited 
number  of  action  verbs.  Since  these  verbs  are  basic  to  his  specialty,  the  user  must  learn  the  meaning  of  these 
verbs  if  he  does  not  already  know  them.  The  meaning  of  each  noun  is  provided  by  the  keyed  pictorial 
locators  - no  matter  how  many  letters  there  are  in  each  noun.  No  sentence  is  over  10  words.  Under  these 
conditions,  the  results  of  reading  index  applications,  which  arc  based  only  on  word  and  sentence  lengths, 
become  meaningless  when  applied  to  these  instructions. 

Format 

A properly  developed  and  utilized  Tl&A,  as  well  as  standard  language  and  keyed  instructions,  are 
necessary  ingredients  of  the  FPJPA  technology.  Using  these  as  foundations  and  considering  the  job 
environment  in  which  the  identified  tasks  were  to  be  accomplished,  several  appropriate  formats  have  been 
developed  to  date.  But  an  important  item  to  remember  is  that  the  necessary  content  of  the  aids  is  far  more 
important  than  the  format. 

A format  similar  to  that  developed  for  the  PIMO  project  has  usually  been  used  for  non-TS  tasks  at  the 
O level  of  maintenance.  Instructions  have  been  formatted  in  a rather  small  booklet  with  instructions  on  the 
left-hand  page  and  the  locator  pictorials  on  the  right-hand  page.  Some  applications  have  provided  pictorial 
foldouts,  each  applied  to  more  than  one  page  of  instructions. 

There  has  been  only  one  development  of  FPJPA  for  the  I level  of  maintenance  for  non-TS  tasks:  the 
AFHRL  aids  for  the  AN/APN-147  and  the  AN/ASN-35.  These  aids  were  similar  to  O level  aids  except  that 
they  were  formatted  in  standard  TO-size  books  (8  by  \0'A  inch  pages). 

There  have  been  two  formats  used  for  FPJPA  for  TS  tasks,  one  in  which  the  instructions  have  been 
placed  in  blocks  and  the  blocks  connected  by  flow  lines  to  indicate  branching,  similar  to  Logic  Tree 
Troubleshooting  Aids  (LTTA),  and,  the  other,  in  which  the  instructions  have  been  placed  in  a scrambled 
book.  The  scrambled  book  format  produces  a document  that  looks  very  similar  to  FPJPA  for  non-TS  tasks. 
The  branching  in  the  scrambled  book  is  provided  by  directions  to  go  to  the  page  on  which  the  instructions 
are  given.  (This  format  provides  for  much  simpler  updating  than  the  blocked  LTTA  format.)  The  draft 
specification  found  in  AFHRL-TR-73-43(I)  (Joyce  et  al.,  1973a)  provides  for  only  the  scrambled  book 
format.  The  earlier  draft  specification  found  in  AFHRL-TR-7 1-53(1)  (see  Folley,  Joyce,  Mallory,  & 
Thomas,  1971a)  provided  for  a choice  of  the  scrambled  book  or  the  logic-tree-like  format. 

Future  applications  of  the  FPJPA  technology  may  result  in  other  formats  that  will  better  meet  the 
needs  of  some  users.  The  point  is  that  the  FPJPA  technology  is  not  limited  by  format,  nor  is  it  truly 
represented  by  format.  For  it  is  possible  to  apply  any  of  the  formats  mentioned  here,  without  the  content 
control  of  the  Tl&A,  but  such  applications  cannot  legitimately  be  considered  true  implementations  of  the 
FPJPA  technology. 
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Validation,  Verification  and  Warranty  of  Maintenance  Manuals 

Validation  by  the  contractor,  followed  by  verification  by  the  Air  Force,  has  long  been  part  of  TO 
policy.  In  some  cases  these  requirements  were  only  table  top  inspections  of  the  final  products.  However,  in 
the  past,  with  the  exception  of  corrections  for  obvious  errors  in  directions  and  format,  little  could  be 
accomplished  by  these  processes  to  insure  the  job  effectiveness  of  maintenance  manuals.  Several 
applications  of  the  FPJPA  technology  have  indicated  that,  no  matter  how  carefully  the  TI&A  was 
accomplished  and  monitored  and  no  matter  how  carefully  the  resulting  criteria  were  applied,  a very  careful 
“hands  on”  validation  and  verification  of  the  results  are  required.  For  maximum  effectiveness,  one  of  these 
efforts  should  include  “hands  on”  equipment  tryout  of  directions  by  members  of  the  target  populations  for 
which  the  directions  have  been  developed. 

Another  safeguard  which  was  applied  in  developing  aids  for  the  C-141  aircraft  was  a 3-year  warranty 
by  the  contractor  of  his  product.  To  insure  maximum  effectiveness  of  the  FPJPA  technology,  such 
warranties  should  be  obtained  for  all  future  applications. 

Applications  of  the  FPJPA  Technology 

Since  some  recent  applications  of  the  FPJPA  technology  have  been  mixed  with  the  application  of 
enriched  LTTA  and  FORECAST  concepts,  a discussion  of  these  concepts  will  precede  the  discussions  of  Air 
Force,  Army  and  Navy  applications  of  the  FPJPA  technology  and  other  maintenance  manual  improvement 
efforts.  In  such  discussions  in  this  appendix,  a maintenance  manual  development,  to  be  considered  an 
implementation  of  the  FPJPA  technology,  must  include  the  proper  development  and  utilization  of  TI&A 
and  must  include  the  use  of  the  standard  language  outlined  in  this  section. 

Enriched  Logic  Tree  Troubleshooting  Aids  (LTTA) 

As  indicated  in  Table  1 of  this  report,  enriched  LTTA  was  one  type  of  TS  data  evaluated  as  part  of 
the  1973  advanced  development  project  1194  concerning  FPJPA.  LTTAs  for  the  0 level  of  maintenance  for 
the  Doppler  radar,  AN/APN-147,  and  its  computer,  AN/ASN-35,  had  already  been  developed  as  part  of  the 
C-141  data  improvement  effort.  These  already  available  O level  LTTAs  were  used  for  project  1194.  Only 
the  LTTA  for  the  intermediate  level  of  maintenance  was  procured  by  AFHRL. 

Both  the  O and  I level  LTTAs  differed  from  traditional  LTTA  in  three  important  respects.  One,  the 
contract  work  statements  called  for  “front  end”  analyses,  very  similar  to  the  Tl&A  required  by  the  FPJPA 
technology  used  for  developing  the  FPTA  for  the  same  project.  These  analyses  assured  that  all  trouble 
signatures  were  identified.  Two,  there  were  in-process  reviews  by  an  Air  Force  maintenance  team  of  these 
analyses,  as  well  as  of  the  resulting  logic  trees.  Three,  the  logic  trees  were  greatly  enriched  by  the  inclusion 
of  a large  amount  of  directions  and  information  at  each  decision  point  in  the  logic  trees. 

An  important  difference  between  the  FPTA,  developed  with  FPJPA  technology,  and  these  LTTA 
developments  was  the  difference  in  specification  and  control  of  the  amount  of  enrichment  to  be  added  to 
the  basic  logic  trees.  For  the  FPTA  development  the  amount  and  kind  of  enrichment  was  controlled  by 
using  the  draft  specification  in  AFHRL-TR-7343(1)  (Joyce  et  al.,  1973a).  Since  no  LTTA  specification  was 
applied  to  either  the  O or  I level  LTTA,  the  amount  of  enrichment  was  left  to  the  discretion  of  each 
developer.  The  developer  of  the  O level  LTTA  who  had  previously  developed  FPTA,  included  pictorial 
locators  as  part  of  the  LTTA  directions  and  information.  However,  the  I level  LTTA  did  not  contain  such 
pictorials.  And,  in  general,  the  O level  LTTA  directions  and  information  were  more  detailed  (enriched).  In 
fact,  they  approached  the  level  of  detail  of  the  FPTA.  Since  the  Air  Force  guidance  provided  to  both 
developers  was  very  similar,  these  two  LTTA  developments  provide  an  example  of  the  variance  in  LTTA 
content  which  can  be  expected  from  different  LTTA  developers  when  the  amount  and  kind  of  enrichment 
is  not  specified. 

In  this  regard,  AFLC  recently  prepared  a request  for  personnel  research  (RPR  77-19)  entitled 
“Development  of  Specifications  for  Maintenance  Task  Analysis  and  Logic  Tree  Troubleshooting  Aids.”  This 
RPR  asks  that  AFHRL  develop  two  specifications:  one,  for  “front  end”  analyses  which  will  be  applicable 
for  both  non-TS  Job  Guide  Manuals  and  LTTA  and  the  other,  a format  specification  for  LTTA.  When 
developed,  the  combination  of  these  specifications  should  result  in  more  consistent  enrichment  of  LTTA. 
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APPENDIX  C:  ARMY  FORECAST  PROJECT 


FORECAST  was  a Human  Resources  Research  Organization  (HumRRO)  project  supported  by  the 
Army  and  directed  by  Dr.  Edgar  L.  Shriver  (see  Shriver  et  al.,  1964).  The  primary  objective  of  the  study 
was  to  shorten  training  time  by  restructuring  the  Army  M-33  fire  control  technician  job.  This  was 
accomplished  by  simplifying  the  performance  of  maintenance  job  (especially  TS  tasks)  with  improved 
maintenance  guidance  and  the  application  of  TOT  program  tailored  to  the  modified  job.  As  such,  it  was  the 
first  IMG&I/TOT  trade-off  study. 

The  development  of  the  original  FORECAST  aids  required  a functional  analysis  of  the  hardware  for 
which  aids  were  developed.  This  analysis  resulted  in  a block  diagram  showing  signal  or  information  flow. 
The  aids  called  “cue  response  sets”  provided  and  emphasized  normal  input  and  output  information  for  each 
block.  Tire  more  recent  modification  of  such  aids  recommended  by  Shriver  and  Hart  (1975)  and  specified 
in  Army  draft  Mil-M-632XX,  Part  I has  added  pictorial  locators  for  input  hardware  test  points. 

Evaluative  Statement 

Although  the  effectiveness  of  such  aids  is  supported  by  a carefully  conducted  and  well  reported 
controlled  study  (Shriver  et  al.,  1964),  there  are  no  hard  data  comparing  the  effectiveness  of  FORECAST 
TS  aids  with  FPJPA  or  LTTA.  The  author  estimates  that  the  FPJPA  for  TS  is  somewhat  more  effective  for 
newly  assigned  first  term  enlistees.  This  estimate  is  based  on  the  assumption  that  the  FPJPA  for  TS 
relegates  more  to  the  “book”  and  less  to  the  “head”  than  do  the  FORECAST  aids.  But  there  are  no 
comparative  hard  data  to  support  this  estimate.  At  the  time  the  Shirver  and  Hart  (1975)  recommendations 
for  TS  aids  were  made  for  the  Army,  the  results  of  the  AFHRL  FPJPA  study  (Potter  & Thomas,  1976) 
were  not  available. 
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APPENDIX  D:  AIR  FORCE  R&D  EFFORTS  CONTRIBUTING 
TO  FPJPA  TECHNOLOGY 


1961  - 1977  AFHRL  Exploratory  Development  Efforts 

Key  Products 

1 . Maintenance  Task  Simulator  (Elliott  & Folley,  1964;  Elliott,  1967b,  1967c) 

2.  Field  Survey  of  Maintenance  Practices  (Folley  & Elliott,  1967) 

3.  Americanization  of  Vietnamization  Specification  and  Handbooks  (Folley  et  al.,  1971a,  1971b; 
Joyce  et  al.,  1971) 

4.  Refinement  of  Americanized  Draft  Specification  and  Handbooks  (Joyce  et  al.,  1973a,  1973b, 
1973c) 

5.  A Proposal  for  a Modified  Technical  Order  System  (Foley,  1975) 

6.  Comparative  R&D  (see  results  below) 

7.  Survey  of  Acceptance  and  Usability  of  Job  Guides  for  C-141  Maintenance 

Results  of  Comparative  R&D 

1.  High  school  subjects  with  12  hours  of  training  using  FPJPA:  high  aptitude  subjects  (80  thru  95 
percentile)  found  93  percent  of  troubles  and  medium  aptitude  subjects  (50  thru  65  percentile)  found  83 
percent  of  troubles  (Elliott,  1967a). 

2.  Air  Force  experienced  subjects  using  TMM  found  84.2  percent  of  troubles  (Elliott  & Joyce, 
1968). 

Results  of  Job  Guide  Survey  (Johnson,  Thompson,  & Martin,  1977) 

1.  Results  of  interviews,  questionnaires  and  observations  indicate  that  C-141  job  guide  program  is 
successful. 

1964  - 1969  AFSC  presentation  of  Information  for  Maintenance 
and  Operation  (P1MO)  Project 

Key  Products 

1.  PIMO  type  FPJPA  for  O level,  non-TS  Tasks  for  C-141  Aircraft  (AFSC-TR-69-155  Vols  I VIII) 

2.  Modified  SIMS  for  O level  TS  tasks  for  C-141 

3.  Comparative  Study:  FPJPA  (non-TS  tasks)  vs  TO;  SIMS  (TS  tasks)  vs  TO. 

Results  of  Comparative  R&D  (AFSC-TR-69-155(I)) 

(All  high  aptitude  subjects) 

1.  Apprentices  and  experienced  Air  Force  specialists  performed  non-TS  tasks  with  no  measurable 
errors  when  using  FPJPA.  Both  made  errors  using  TO.  Apprentices  could  not  complete  many  non-TS  tasks 
using  TO. 

2.  Experienced  Air  Force  specialists  reduced  TS  errors  by  92  percent  when  using  modified  SIMS 

aids. 
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1969  - 1972  USAF  Vietnamization  Project 


Key  Products 

1.  P1MO  type  FPJPA  for  three  hardware  systems  for  0 level  TS  and  non-TS  tasks;  also  P1MO  type 
SIMS  TS  Aid  for  one  system. 

2.  Converted  FPJPA  concept  to  technology  — Mil-J-83302,  handbooks  for  JPA  developers  and 
managers. 

3.  Applied  this  technology  to  seven  hardware  systems  (O  level  TS  and  non-TS  tasks)  (Appendix  F). 

4.  Articulated  cuing  requirement  for  FPJPA  directions  AFHRL-TR-75-38  (Shriver,  1975). 

5.  Evaluation  of  UH-1H  Aids  (Shriver  & Foley,  1975). 

1967  — 1972  First  AFHRL  Advanced  Development  Effort 

Key  Products 

1 . Pre-specification  FPJPA  for  TS  (AN/APN-147  and  AN/ASN-35) 

2.  SIMS  type  TS  aids  (AN/APN-147  and  AN/ASN-35) 

3.  Programmed  Instruction  on  use  of  SIMS  aids 

4.  Demonstration  of  FPJPA/TOT  trade-off  (Mullen  & Joyce,  1974) 

5.  Plan  for  comparative  controlled  R&D 

Plan  for  Comparative,  Controlled  R&D 

1.  Multi-variant  design  considering  FPJPA  vs.  SIMS  vs.  TO;  high  vs.  medium  aptitude;  traditional 
training  and  TOT  trade-off  with  FPJPA;  and  transferability  from  TOT/FPJPA  on  one  hardware  to  FPJPA 
on  other  hardware.  (Funds  withdrawn  - project  cancelled  1972.) 

1973  - 1976  Second  AFHRL  Advanced  Development  Effort 

Key  Products 

1.  FPJPA  for  both  0 and  I level  TS  and  non-TS  Tasks  (FPJPA  for  TS  tasks  called  FPTA) 

2.  LTTA  for  0 level  TS  tasks  with  pictorial  locators  (Borrowed  from  C-141  TO  improvement 
program) 

3.  Phase  I R&D  — comparative  study 

4.  Phase  II  R&D  - Plan  for  FPJPA/TOT  trade-off  study 

Results  for  Phase  I Comparative  R&D  (High  Aptitude  Subject  only) 

1.  FPTA  vs.  LTTA  vs.  TO  - See  Tables  1,2,3,  and  4.  (Potter  & Thomas,  1976;  Potter  et  al.,  1976a, 
1976b.) 

Plan  for  Phase  II  Comparative  R&D  (High  Aptitude  Subjects  only) 

1.  Design  considered  only  TOT  tailored  for  FPJPA  plus  FPJPA  vs.  TOT  tailored  for  TO  plus  TO. 
Design  did  not  consider  medium  aptitude  subjects  or  transferability  from  TOT/FPJPA  trade-off  on  one 
hardware  to  FPJPA  on  other  hardware.  (Funds  withdrawn;  project  cancelled.) 
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APPENDIX  E:  SUMMARY  OF  CONTROLLED  COMPARATIVE  TRYOUTS 
CONCERNING  FPJPA  TECHNOLOGY  FOR  NON-TS  TASKS 


The  most  extensive  controlled  tryout  of  FPJPA  for  non-TS  tasks,  which  contributed  to  the 
technology,  was  made  as  part  of  the  P1MO  project.  In  addition,  the  Navy  and  Army  have  conducted  less 
extensive  tryouts  of  similar  data. 

The  P1MO  Project 

The  PIMO  project  included  two  early  exploratory  development  R&D  efforts  and  an  advanced 
development  operational  R&D  effort.  A complete  description  of  the  project  and  its  results  are  found  in  the 
eight  volumes  of  AFSC-TR-69-155.  One  of  the  exploratory  efforts  and  the  advanced  development  effort 
were  conducted  under  contract  for  the  Air  Force  by  Serendipity,  Inc.,  of  Chatsworth,  California.  The 
principal  investigator  was  Dr.  Kay  Inaba,  now  Preseident  of  XYZYX  Information  Corporation  of  Canoga 
Park,  California.  This  project  was  managed  by  the  Ballistics  Systems  Division  (now  the  Space  and  Missile 
Systems  Office)  of  Air  Force  Systems  Command  (AFSC).  Although  not  an  AFHRL  project,  the  Advanced 
Systems  Division  of  AFHRL  was  represented  on  the  PIMO  working  group  from  its  start  in  1964.  The 
project  was  completed  in  1969.  From  a money  point  of  view,  this  one  project  was  a much  larger  effort  than 
all  of  the  AFHRL  efforts.  The  PIMO  project  cost  $2,700,000. 

The  PIMO  advanced  development  R&D,  as  planned,  was  to  include  the  development  and  operational 
evaluation  of  advanced  type  job  performance  aids  for  both  O and  I level  maintenance  tasks  for  the  C-141A 
aircraft.  Fully  proceduralized  aids  were  to  be  developed  for  all  tasks  except  electronic  troubleshooting  tasks 
and  engine  tasks.  SIMS  type  aids  were  to  be  used  for  troubleshooting.  Tire  FPJPA  were  to  be  presented  in 
both  a hardback  or  book  mode  and  a rapid  access  audiovisual  mode. 

A large  scale  field  evaluation  was  planned  to  compare  the  relative  maintenance  effectiveness  for 
non-TS  tasks  of  traditional  Air  Force  maintenance  manuals,  hardback  FPJPA  and  audiovisual  FPJPA.  Due 
to  a cutback  in  funding,  the  aids  for  I level  maintenance  tasks  were  not  developed. 

This  experiment  was  originally  designed  for  obtaining  systems  effectiveness  infomiation  using  gross 
etfectiveness  measures,  such  as  down  time  and  turnaround  time  using  only  those  maintenance  tasks  that 
presented  themselves  in  the  operational  flight  line  environment.  This  design  proved  to  be  unworkable.  One 
of  the  problems  was  that  tasks  could  not  be  matched  for  the  various  treatments  of  subjects.  As  a result,  a 
controlled  study  based  on  a counterbalanced  design  was  used.  Like  non-TS  tasks  were  performed  by  each 
subject  in  the  control  and  experimental  groups.  The  results  of  this  study  are  summarized  in  the  following 
paragraphs. 

1.  Essentially  no  difference  in  performance  time  for  experienced  specialists  performing  with  FPJPA 
aids  and  for  those  performing  with  conventional  T.O.  aids  and  information.  These  experienced  specialists 
averaged  25  years  of  age  and  approximately  19.2  months  experience  (5  and  7 skill  levels). 

2.  These  experienced  specialists  made  errors  when  performing  with  traditional  maintenance  manuals. 
No  errors  were  made  when  these  specialists  used  FPJPA. 

3.  The  apprentices  performing  with  PIMO  aids  required  slightly  less  time  on  the  average  to  complete 
the  maintenance  tasks  than  the  experienced  specialists  performing  with  PIMO  aids.  These  apprentices 
averaged  20  years  of  age  and  approximately  5 months  exposure  to  the  C-141  aircraft,  but  they  had  no 
previous  experience  on  the  tasks  used  in  the  experiment. 

4.  No  errors  were  made  by  the  apprentices  when  using  FPJPA  aids. 

5.  These  same  apprentices  required  approximately  a third  more  time  to  perform  the  tasks  using 
traditional  maintenance  manuals  and  made  more  errors.  These  apprentices  used  the  TO  aids  and 
information  after  they  had  performed  the  tasks  several  times  using  FPJPA  aids.  (Surprisingly,  they  made 
fewer  errors  than  the  experienced  specialists  using  traditional  maintenance  manuals.) 
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6.  A second  group  of  apprentices  was  allowed  to  use  only  traditional  maintenance  manuals.  This 
group  required  much  more  time  and  made  many  more  errors  than  either  the  experienced  specialists  or  the 
apprentices  in  the  comparative  experiment.  In  two  of  the  three  jobs,  these  apprentices  were  unable  to 
complete  the  jobs. 

7.  A group  of  individuals  from  unrelated  career  fields  was  also  given  the  same  tasks  to  perform  using 
FPJPA.  Their  performance  was  similar  to  that  of  the  apprentices  described  in  paragraph  3 and  4.  They  were 
able  to  perform  the  tasks  very  well  with  no  errors. 

8.  No  significant  difference  was  found  between  audiovisual  and  visual  (booklet)  presentation  of 

data. 

Although  the  actual  development  of  the  P1MO  FPJPA  for  non-TS  tasks  was  preceded  by  a systems 
analysis  and  a so-called  task  analysis,  these  analyses  were  not  as  stringent  as  those  now  required  by  current 
technology  (AFHRL-TR-73-43(I),  Joyce  et  al.,  1973a).  In  addition,  these  data  were  never  given  complete 
“hands  on”  validation  and  verification.  So,  the  data  would  not  meet  the  current  requirements  of  the  FPJPA 
technology.  However,  the  PIMO  FPJPA  used  for  the  tasks  selected  for  the  controlled  experiment  had  been 
completely  verified  and  probably  would  compare  favorably  with  FPJPA  developed  in  keeping  with  current 
technology. 

An  important  contribution  of  the  PIMO  project  to  the  FPJPA  technology  was  the  standard  language 
for  step-by-step  directions  supported  by  location  pictorials.  Its  controlled  experimental  conditions  provided 
sufficient  and  convincing  hard  data  concerning  the  effectiveness  of  FPJPA  for  non-TS  tasks.  It 
demonstrated  that  experienced  technicians  would  perform  better  maintenance  if  they  used  FPJPA. 
Important  from  the  standpoint  of  better  utilization  of  first  enlistment  personnel,  it  demonstrated  that 
inexperienced  personnel  could  perform  non-TS  tasks  in  a satisfactory  manner  using  FPJPA.  All  of  the 
personnel  used  for  the  PIMO  study  possessed  high  aptitudes. 

Early  Navy  Development  and  Tryout  of  PIMO  Type  FPJPA 

In  the  early  1970’s  the  Naval  Weapons  Engineering  Support  Activity  conducted  a less  extensive 
controlled  evaluation  of  non-TS  PIMO  type  FPJPA  which  were  developed  for  an  Auxiliary  Power  Unit  — 
System  No.  NCA-8  under  contract  by  XYZYX  Information  Corporation.  The  maintenance  concept  for  this 
power  unit  required  both  electrical  and  engine  specialists.  During  the  study  both  types  of  the  specialists 
were  required  to  perform  both  electrical  and  engine  tasks.  An  engine  specialist  was  considered  to  be 
experienced  subject  on  engine  tasks  and  an  inexperienced  subject  on  electrical  tasks  and  vice  versa.  The 
results,  reported  by  Horne  (1972),  are  very  similar  to  those  of  the  Air  Force  PIMO  project. 

1.  Inexperienced  subjects  using  FPJPA  made  no  more  errors  than  experienced  subjects  using 
traditional  maintenance  manuals. 

2.  The  performance  of  experienced  subjects  was  error  free  when  using  FPJPA.  But  they  required 
more  time  than  when  using  traditional  maintenance  manuals. 

3.  Before  the  FPJPA  produced  by  contractor  could  be  used  effectively  by  the  inexperienced 
subjects,  they  required  complete  “hands  on”  validation. 

In  addition,  the  Naval  Air  Systems  Command  had  FPJPA  developed  for  four  different  subsystems, 
each  by  one  of  four  different  contractors.  These  were  developed  under  the  Air  Force  Vietnamization 
specification  Mil-J-83302  and  its  accompanying  handbooks  (Applied  Science  Associates,  1971a,  1971b). 
However,  the  Navy  did  not  have  the  facilities  for  adequate  in-process  review  and  acceptance  of  the  products 
of  TI&A  and  to  insure  that  these  products  were  used  as  criteria  during  FPJPA  development.  Although  each 
contractor  delivered  its  FPJPA,  they  were  never  given  a field  evaluation. 

Early  Army  Development  of  PIMO  Type  FPJPA 

Also,  in  the  early  1970’s,  the  Letterkenny  Army  Depot  at  Chambersburg,  Pennsylvania,  developed 
FPJPA  for  six  small  itgms  of  equipment  on  an  in-house  basis.  The  systems  included  an  armament  pod  for 
aircraft,  a search  light,  a storage  battery,  a tank  engine,  a gasoline  engine  water  pump,  and  a tow  launcher. 
For  a number  of  reasons,  the  evaluation  of  these  aids  was  never  completed. 
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APPENDIX  F:  SUMMARY  OF  USAF  JOB  PFRFORMANCE  AID 
PROJECT  FOR  VIETNAMIZATION 


As  part  of  the  Vietnamization  program,  job  performance  aids  (JPA)  were  developed  to  support 
maintenance  of  several  hardware  systems  by  maintenance  personnel  of  the  Vietnamese  Air  Force  (VNAF). 
Fully  proceduralized  job  performance  aids  (FPJPA)  were  developed  for  both  TS  and  non-TS  tasks.  These 
aids  were  first  developed  for  the  UH-1H  helicopter.  In  addition,  SIMS  type  TS-aids  were  developed  for  this 
aircraft.  Table  FI  indicates  the  R&D  sources  of  these  different  aids. 


Table  FI.  Indicating  the  Immediate  Sources  of  JPA  Data  Selected  for  the  Support 
of  VNAF  Organizational  Maintenance  of  the  UH-1H 


Sources 

Types  of  Job  Performances 

PIMO  Project 

AFHRL  Program 

Aid  (JPA)  Data 

1964—1969 

1961  —Continuing 

1 . Fully  Proceduralized  Job  Performance 

Aids  (FPJPA)  for  Non-Troubleshooting 

Tasks  (Job  Guide  Manuals) 

X 

X 

2.  FPJPA  for  Troubleshooting  Tasks 
(called  Fully  Proceduralized 

Troubleshooting  Aids-FPTSA) 

X 

3.  Maintenance  Dependency  Chart 
(MPC)  Centered  Troubleshooting 

Aids 

X 

Adequate  specifications  were  not  available  for  producing  these  UH-1H  aids  and  the  deadline  for 
producing  the  UH-1H  aids  was  very  short.  As  a result,  a sole-source  contract  was  given  to  Dr.  Kay  Inaba’s 
firm,  the  XYZYX  Information  Company.  He  had  been  the  principal  investigator  on  the  PIMO  Project  and  it 
appeared  that  he  had  the  capability  of  meeting  the  short  deadline. 

While  the  UH-1H  JPA  were  being  developed,  other  JPA  developmental  efforts  were  directed  for  the 
VNAF.  But  before  JPA  could  be  procured  on  a multi-source  basis,  an  adequate  specification  was  necessary. 
A contract  was  let  with  Applied  Science  Associates,  Inc.,  (ASA)  of  Valencia,  Pennsylvania,  for  a 
specification  (Mil-J-83302)  and  companion  guidance  handbooks  (Applied  Science  Associates,  1971a, 
1971b).  These  documents  were  later  Americanized  and  published  as  a three-volume  technical  report, 
AFHRL-TR-71-53  (Folley  et  al„  1971,  1971b;  Joyce,  Folley,  & Elliott,  1971).  These  Americanized 
documents  have  since  been  carefully  updated  and  published  by  AFHRL  as  another  three-volume  technical 
report,  AFHRL-TR-7343  (Joyce  et  al„  1973a,  1973b,  1973c). 

All  of  the  Vietnamization  efforts  are  listed  in  Table  F2  including  these  guidance  documents  and  the 
evaluation  of  the  UH-1H  JPA.  While  the  specification  and  guidance  document  were  being  developed, 
additional  JPA  were  being  developed  on  a sole-source  basis  for  two  aircraft  - the  C-123K  and  the  CH47A. 
The  other  follow-on  efforts  (with  the  exception  of  the  Seek  Point  project)  were  procured  on  a multisource 
basis.  Table  F2  indicates  the  time  frame  of  each  of  the  Vietnamization  efforts,  the  types  of  JPA  included, 
and  the  levels  of  maintenance  covered.  All  of  these  aids  produced  for  Vietnamization  were  in  a 
two-language  format. 


L 


47 


XXX  U U c £ c tLtl' 

>•  > >•*  * .S  2 2 2 ,5  .5  . 

y ^ ^ ^ ^ ^ « C C C s W 

/■  > >■  V)  V)  W « O0>O4»W4> 

x x x < < < s £ x u u & & i 


— ©_*-*  — L'NMfSNNNn^ 
i — r~-  t — t-"  ^ i f^-r^c — r~—  r — c — i — i 

**  k-  H u _ U,  _ •?  CJjOt)D^aiiOtiCOjUj>> 
oa^oSao^aaasssSfa 

OCSQ«5<Ow<<<<<<<S 


tv  r*  1 ' f~-t~'t~-r~-r^-r~-r~r^ 

£ o o §*  5*  Jj  Js  ootloooo^' 
.3  O O £ £ 5 S ******** 


OOOOOOOc/$ 


xxxxxxx  xxxxxxxx 


X X X X X 


xxxxxxx  xxxxxxxx 


xxxxxxx  xxxxxxxx 


> & 0*  5 
1 Q S '2 


< 2 2 I 

&•  " —>  *j 

- u < .a 
* m c-  ia 
3«t  " 
i -r  a £. 

3 U U t/i 


o o w 
«*-<  t<—  ^ 

o o _ 

22x 

XJ  T3  — 

S Sx 
x a 3 


:<2 
. a.  *—» 

! y CU  £- 
: .S  !c  ^ 

?s?f  *-§ 

!“  «2  J 

; s 1 .s  | .s 

"O  M O " u 

> 2,  m • <U 

* H Q£  *o  O c/i 


^ § 

I X X 


O'  v>  r—  c“>  m © 1 


I 

U 'O  W 


westinghouse  — Westinghouse  Corp., 
Kentron  — Kentron  Hawaii,  Dallas,  T 


The  JPA  developed  for  the  UH-1H  aircraft,  like  those  developed  for  the  C-123K  and  CH-47A  aircraft, 
were  developed  without  the  benefit  of  the  structured  Tl&A  required  by  specification  Mil-J-83302.  A formal 
evaluation  was  made  of  only  the  UH-1H  aids  (see  Shriver  & Foley,  1975).  The  lessons  learned  from  the 
various  procurements  listed  in  Table  F2  coupled  with  those  learned  from  the  evaluation  of  the  UH-1H  aids 
contributed  a great  deal  to  the  evolution  of  the  FPJPA  technology. 

Lessons  Learned  from  Evaluation  of  the  UH-1H  Aids 

1.  The  development  and  evaluation  of  the  UH-1H  JPA  strongly  indicated  that  a traditional  format 
specification  such  as  Mil-M-38800  would  not  produce  complete  and  effective  JPA. 

2.  A systematic  task  identification  procedure  is  required  to  insure  that  a step-by-step  procedure  is 
developed  and  available  for  every  hardware  task  for  which  the  maintenance  technician  is  responsible. 

3.  Technically  correct  step-by-step  directions  in  standard  language,  keyed  to  locator  pictorials,  are 
necessary  but  not  sufficient  attributes  of  task  directions.  To  insure  correct  performance  of  each  task,  the 
directions  and  accompanying  pictorials  for  each  task  must  supply  all  the  necessary  information  and  cues 
required  by  members  of  the  target  population  to  perform  the  task  (Shriver,  1975;  Shriver  & Foley,  1975). 

4.  A systematic  analysis  is  required  for  each  task  to  insure  the  presence  of  all  the  necessary 
information  as  well  as  to  identify  most  of  the  cues  required  for  the  preparation  of  effective  draft  directions 
and  accompanying  pictorial  locators.  Such  draft  directions  must  be  given  a “hands  on"  hardware  validation 
by  the  contractor  to  insure  that  all  necessary  cues  are  present  and  that  a typical  maintainer  can  perform  the 
tasks  using  the  directions.  (In  this  regard,  the  more  careful  the  original  analysis  of  each  task,  the  fewer  the 
modifications  required  for  its  directions  during  validation.) 

An  Expanded  Specification 

The  legal  requirement  and  guidance  for  such  task  identification  and  analysis  (Tl&A)  of  identified 
tasks  was  provided  for  the  multisource  procurements  of  JPA  by  the  specification  (Mil-J-83302)  and  the 
companion  guidance  handbooks  (Applied  Science  Associates,  1971a,  1971b).  These  multisource 
procurements  included  aids  for  the  C-7A  aircraft,  three  jet  engines,  two  fire  trucks,  and  a refueling  vehicle 
(indicated  in  Table  F2).  However,  the  existence  and  callout  of  the  specification  in  the  procurement 
contract  is  a necessary  but  not  sufficient  action  for  the  successful  development  of  JPA.  The  development  of 
the  sub-products  of  Tl&A  for  evaluation  and  acceptance  by  the  Air  Force  was  a new  experience  for  both 
contractor  and  Air  Force  procurement  personnel. 

Key  Lessons  Learned  from  Application  of  Expanded  Specification 

1.  The  JPA  technology  as  structured  in  Mil-J-83302  is  a new  technology  requiring  substantial 
changes  in  skills  and  attitudes  of  technical  writers  and  their  associated  management  personnel -both 
contractor  and  Air  Force.  The  application  of  this  specification  and  its  accompanying  guidance  to  these 
multisource  procurements  confirmed  the  requirement  for  DOD  capability  for  evaluating  and  accepting  the 
TI&A  products,  for  insuring  the  use  of  these  products  as  criteria  during  the  development  of  the  aids,  and 
for  conducting  the  “hands  on”  verification  of  the  completed  JPA. 

2.  Since  most  of  these  actions  were  new  to  both  Air  Force  and  contractor  personnel,  these 
capabilities  were  developed  by  expensive  “cut  and  try”  procedures.  No  doubt,  the  most  effective  action  for 
developing  the  necessary  skills  and  attitudes  would  be  appropriate  training  programs  which  would  provide 
supervised  practice  of  the  skills  required.  Such  training  programs  also  would  help  develop  the  necessary 
attitudes  toward  this  unfamiliar  technology. 

3.  The  Vietnamization  project  had  no  formal  mechanism  for  effectively  utilizing  the  “know  how”  of 
the  R&D  community. 
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APPENDIX  G:  SUMMARY  OF  AIR  FORCE  HUMAN  RESOURCES  LABORATORY  (AFHRL) 
CONTROLLED  COMPARATIVE  STUDY  CONCERNING  THE  EFFECTIVENESS 
OF  FPJPA  TECHNOLOGY  FOR  TS  TASKS 
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This  study  generated  comparative  data  concerning  Fully  Proceduralized  Troubleshooting  Aids 
(FPTA),  enriched  Logic  Tree  Troubleshooting  Aids  (LTTA),  traditional  maintenance  manuals  (TMM) 
(called  Technical  Orders  (TO)  by  the  Air  Force). 

The  detailed  results  are  reported  in  AFHRL-TR-76-74(I)  (Potter  & Thomas,  1976).  Table  G1 
indicates  success  in  terms  of  percentage  of  troubles,  identified  by  each  experience  level  using  each  type  of 
maintenance  data.  All  of  the  subjects  of  this  study  had  high  electronic  aptitudes  (80  percentile  and  above). 

Table  Gl.  Comparison  of  FPTA,  LTTA,  and  TO  in  Terms 
of  Percentage  of  Troubles  Found 


Type  of  Date 


Experience 

of 

Subjects 

Maintenance  Level 

FPTA 

LTTA 

TO 

O 

1 

O 

1** 

O 

1 . 

Inexperienced 

Organizational  (0) 

88.9 

77.8 

* 

Intermediate  (I) 

88.8 

60.6 

* 

Through  6 months 

0 

100 

100 

100 

I 

95.4 

89.4 

60.6 

Over  6 months 

0 

95.8 

95.8 

100 

I 

95.4 

87.8 

78.8 

*No  data  collected. 

**I  level  LTTA  did  not  contain  location  pictorials. 


Table  G2  indicates  the  excessive  number  of  spare  parts  utilized  in  the  process  of  finding  these  faults. 
These  data  are  expressed  in  terms  of  the  average  number  of  unnecessary  spare  parts  used  per  trouble.  In  this 
table  the  results  for  the  two  experienced  groups  are  combined. 


Table  G2.  Comparison  of  FPTA,  LTTA,  and  TO 
in  Terms  of  Spare  Parts  Used 


Experience 

of 

Subjects 

Maintenance 

Level 

Types  of  Data 

FPTA 

LTTA 

TO 

O 1 

O 1 

O 1 

Inexperienced 

0 

.77 

1.00 

i 

1.44 

2.22 

Experience 

0 

.13 

.25 

.53 

I 

1.03 

.94 

1.86 

Table  G3  indicates  the  developmental  cost  in  dollars  of  the  FPTA  and  LTTA  for  both  O and  I 
maintenance  for  the  AN/APN-147  and  AN/ASN-35.  The  FPTA  was  more  expensive  to  develop  than  LTTA. 
(It  should  be  recalled  that  the  LTTA  for  I maintenance  did  not  contain  pictorial  locators  which,  if  added, 
would  no  doubt  have  increased  the  cost). 
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Table  G3.  Comparison  of  Cost  Experienced  in  Dollars 
for  the  Development  of  FPTA  and  LTTA 
for  the  AN/APN-147  and  AN/ASN-35 


Maintenance  Level 

FPTA 

LTTA 

Difference 

Organization  (O) 

14,451 

7,593 

6,858 

Intermediate  (I) 

164,680 

111,076 

53,604 

Total 

179,131 

118,669 

60,462 

When  the  results  presented  in  these  three  tables  are  considered  in  terms  of  what  type  of 
troubleshooting  aids  are  most  cost  effective  for  the  type  of  experienced  personnel  now  assigned  to  the  Air 
Force , one  can  easily  conclude  that  the  LTTA  is  the  most  cost  effective  in  terms  of  TO  development 
dollars. 

Jt.  At  the  O level  of  maintenance,  experienced  personnel  did  quite  well  with  TO,  LTTA,  and  FPTA. 

2.  At  the  1 level,  both  LTTA  and  FPTA  were  far  superior  to  TO. 

3.  At  the  1 level,  FPTA  was  somewhat  more  effective  than  LTTA  in  terms  of  troubles  found.  And  in 
terms  of  unnecessary  spare  parts  used,  both  were  about  the  same  with  the  slight  difference  favoring  LTTA. 
But  in  terms  of  cost,  the  FPTA  was  $53,604  (or  48  percent)  more  costly  to  develop. 

At  least  for  the  present,  the  Air  Force  Logistics  Command  (AFLC)  technical  data  establishment  has 
decided  in  favor  of  the  LTTA.  But  such  a decision  must  be  based  on  tacit  assumptions  that  the  personnel 
using  these  aids  will  always  have  the  high  aptitudes,  the  long  training  times,  and  the  substantial  experience 
advantages  now  enjoyed  by  the  Air  Force. 

However,  when  the  results  presented  in  these  tables  together  with  other  available  data  are  considered 
in  terms  of  potential  for  more  effective  utilization  of  first  enlistment  personnel,  one  can  conclude  that  a 
large  amount  of,  the  potential  for  effecting  substantial  future  personnel  savings  may  have  been  lost  by  the 
decision  in  favor  of  LTTA.  The  following  results  obtained  from  inexperienced  personnel  indicate  that  such 
personnel  performed  much  better  with  FPTA. 

1.  Inexperienced  subjects  found  more  O and  I level  troubles  when  using  FPTA  than  when  using 
LTTA  - at  the  0 level,  11.1  percent  more  (88.9  — 77.8),  and  at  the  I level,  28.2  percent  more  (88.8  — 
60.6).  (Inexperienced  subjects  did  not  use  TO.) 

2.  Inexperienced  subjects  used  fewer  unnecessary  spare  parts  per  trouble  when  using  FPTA  than 
when  using  LTTA  for  both  levels  of  maintenance,  i.e.,  .23  more  at  the  O level  and  .78  more  I level. 

These  data  would  indicate  that  by  using  FPTA,  the  Air  Force  could  get  immediate  utilization  of 
newly  assigned  personnel  either  from  school  or  from  assignments  on  other  hardware.  This  would  greatly 
reduce  the  requirement  for  expensive  on-the-job  training  (OJT)  or  cross-training.  It  would  also  provide  a 
more  rapid  buildup  potential  in  case  of  national  emergencies.  When  viewed  from  the  potential  savings  in 
personnel  costs  and  potential  for  rapid  personnel  buildup  points  of  views,  the  relatively  small  immediate 
savings  of  $58,207  (from  Table  G3,  $118,669  — $60,462)  could  be  very  expensive  in  the  long  run.  Also, 
the  use  of  LTTA  in  lieu  of  FPTA  could  greatly  reduce  the  potential  savings  from  well  designed  FPJPA/task 
oriented  training  trade-offs. 

Within  the  next  10  years  the  number  of  young  people  in  the  United  States  population  is  expected  to 
decrease  rapidly.  If  for  no  other  reason,  it  is  most  probable  that  DOD  will  have  to  lower  its  aptitude 
requirements  for  entry  into  maintenance  jobs.  Although  the  1967  advanced  development  plan  called  for 
consideration  of  medium  aptitude  personnel,  it  was  not  included  in  the  study  reported  here.  But  a previous 
exploratory  development  study  indicated  that  inexperienced,  average  aptitude  subjects  using  FPTA  found 
about  10  percent  fewer  troubles  on  the  first  attempt  than  did  high  aptitude  subjects.  Projecting  this 
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difference  on  the  data  presented  in  Table  Cl,  one  could  expect  average  aptitude  subjects  to  find  about  80 
percent  of  the  troubles  using  FPJPA  for  both  0 and  1 levels.  If  FPJPA  were  developed  for  average  aptitude 
subjects  as  the  target  population,  this  percentage  could  probably  be  increased.  In  any  case,  FPTA  would  be 
more  effective  than  LTTA,  for  maximum  utilization  of  average  aptitude  (40  to  70  percentile)  personnel. 

In  defense  of  the  Air  Force  TO  establishments,  currently  a prime  criterion  for  their  performance  is 
economy  in  TO  production.  Tire  TO  people  get  no  credit,  financial  or  otherwise,  for  saving  personnel 
dollars.  And  there  is  no  mechanism  for  transferring  personnel  money  to  TO  production. 
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APPENDIX  H:  SUMMARY  OF  NAVAL  AIR  DEVELOPMENT  CENTER  (NADC) 
FPJPA/TOT  TRADE-OFF  STUDY  (Theisen  et  al.,  1978) 


Description  and  Results 

1 . Starting  in  1972  the  Human  Engineering  Division  of  the  Naval  Air  Development  Center  (NADC), 
Warminister,  Pennsylvania,  supported  a FPJPA/TOT  trade-off  effort.  Historically,  this  effort  had  its  genesis 
in  the  AFHRL  FPJPA/TOT  trade-off  demonstration  which  was  conducted  at  Altus  AFB,  Oklahoma  in  early 
1972  (see  Mullen  & Joyce,  1974).  Whereas  the  Altus  effort  was  only  a demonstration,  the  Navy  effort  was 
planned  as  a full  scale  controlled  study  to  determine  the  relative  effectiveness  of  the  combined  FPJPA  and 
TOT  technologies  with  respect  to  traditional  theory  based  training  and  TMM.  Unfortunately,  the  controlled 
comparative  aspects  of  this  plan  were  never  completed.  Nevertheless,  FPJPA  for  O and  1 level  maintenance 
of  the  AN/AQA-7  Sonar  Recorder  Group,  as  well  as  matching  TOT  packages  (one  for  O level  technicians 
and  one  for  1 level  technicians),  were  developed.  And  hard  data  as  to  their  effectiveness  were  obtained. 

2.  The  job  aids  for  the  AN/AQA-7  Sonar  Recorder  Group  developed  for  this  effort  reflected  the 
vintage  of  FPJPA  technology  found  in  the  three  volume  technical  report  AFHRL-TR-71-53  (Folley  et  al., 
1971a,  1971b;  Joyce  et  al.,  1971).  This  was  a more  advanced  vintage  of  developmental  technology  than 
reflected  by  the  FPJPA  used  in  the  Altus  demonstration,  which  was  developed  without  the  benefit  of  such 
structured  guidance.  The  aids  used  at  Altus  were  limited  to  TS,  whereas  the  aids  developed  for  this  effort 
covered  non-TS  tasks  as  well.  However,  the  author  feels  that  the  AN/AQA-7  developmental  effort  omitted 
some  very  important  aspects  of  the  specification  and  guidance  documents  found  in  AFHRL-TR-71-53. 
These  documents  require  that  all  the  formal  products  of  the  task  identification  and  analyses  (TI&A)  of 
identified  tasks  be  reviewed  and  accepted  by  the  procuring  agency,  in  this  case  NADC.  They,  also,  require 
that  the  procuring  agency  ensure  that  these  products  are  utilized  as  content  criteria  during  the  development 
of  the  actual  FPJPA.  The  only  IT&A  products  which  received  such  treatment  were  the  troubleshooting 
action  trees.  In  addition,  due  to  funding  and  scheduling  problems  all  FPJPA  did  not  receive  the  required 
contractor  validation  and  the  government  verification.  Only  those  FPJPA  used  for  the  experimental  tryout 
problems  received  such  treatment. 

3.  As  to  the  TOT  packages,  they  were  more  complete  than  those  developed  for  the  Altus 
demonstration.  For  those  subjects  assigned  to  organizational  (O)  maintenance  a 13  week  TOT  package  was 
developed  to  prepare  average  aptitude  personnel  (approximately  40  to  50  percentile)  to  perform  0 level 
maintenance  on  the  AN/AQA-7  Sonar  Recorder  Group  of  the  S2-G  aircraft  using  FPJPA.  A 16  week  TOT 
package  was  developed  to  prepare  the  same  range  of  average  aptitude  personnel  to  perform  I level 
maintenance  tasks  on  the  same  equipment  using  FPJPA.  The  contract  monitor  has  indicated  that  these  TOT 
packages  were  of  good  quality.  However  since  these  packages  arrived  4 weeks  after  training  started,  the 
subjects  did  not  obtain  the  full  advantage  of  their  potential. 

4.  Traditionally,  the  AN/AQA  System  has  been  maintained  by  “A”  school  graduates  using  TMM. 
The  entry  aptitude  for  “A”  school  is  84  percentile  and  above.  The  appropriate  “A”  school  program  was  22 
weeks  in  length  and  was  theory  based.  Students  selected  for  O level  maintenance  received  2 additional 
weeks  of  AN/AQA  specific  training  making  a total  of  24  weeks.  Students  selected  for  I level  maintenance 
received  14  additional  weeks  of  system  specific  training  making  a total  of  36  weeks.  So  the  TOT  package 
for  O level  training  was  1 1 weeks  shorter  than  conventional  training  (a  46  percent  savings  in  training  time) 
and  the  I level  package  was  20  weeks  shorter  than  conventional  training  (a  56  percent  savings  in  training 
time). 

5.  The  original  plan  for  the  conduct  of  TOT  included  the  following  assumptions:  (a)  students  would 
arrive  in  four  groups  of  10  each  approximately  1 to  2 weeks  apart,  (b)  there  would  be  four  instructors,  two 
to  teach  I level  maintenance  program  and  two  to  teach  O level  maintenance  program,  (c)  two  AQA-7 
systems  and  AQM-18  test  benches  could  be  made  available  for  the  exclusive  use  of  the  project,  and  (d) 
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access  to  S2G  aircraft  (with  operating  AQA-7s)  could  be  obtained.  Unfortunately,  none  of  these  conditions 
were  met,  the  students  arrived  early  and  all  within  a single  10-day  period,  only  three  instructors  could  be 
recruited,  only  one  AQA-7  and  AQM-18  could  be  provided,  and  access  to  S2G  aircraft  was  limited  due  to 
heavy  flight  commitments  of  the  operational  forces.  Finally,  as  mentioned  previously,  the  TOT  training 
materials  were  delivered  approximately  1 month  late  and  not  properly  collated. 

After  delivery  and  assembly  of  the  training  materials,  the  41  trainees  were  divided  into  three  sections; 
two  sections  of  13  trainees  each  were  designated  for  “1”  level  training,  and  one  section  of  15  trainees  was 
designated  for  “O”  level  training.  Since  only  one  AQA-7  system  and  one  AQM-18  test  bench  could  be 
made  available  for  instruction,  it  was  decided  to  establish  a 24-hour  teaching  day  consisting  of  three  8-hour 
sessions.  Sessions  were  scheduled  Monday  through  Thursday  to  conform  with  the  normal  NAS  Cecil  Field, 
Florida,  work  week.  Arrangements  were  made  with  the  Aviation  Intermediate  Maintenance  Department  of 
NAS,  Cecil  Field  to  obtain  priority  repairs  on  the  AQA-7  components  used  in  training.  Despite  this 
excellent  support,  many  training  hours  were  irretrievably  lost  due  to  equipment  being  inoperative  for 
maintenance.  Consequently,  the  trainees  received  much  less  practice  than  planned  because  of  the  larger 
than  planned  class  size  and  the  shortage  of  equipment. 

6.  The  evaluation  of  the  subjects  sought  to  provide  a realistic  and  practicable  work  setting  with 
representative  maintenance  problems.  Since  the  data  collection  had  to  be  accomplished  in  4 weeks,  only 
five  problems  could  be  scheduled  for  each  O level  subject  and  only  four  for  each  I level  subject.  However, 
the  problems  selected  are  considered  representative.  The  results  are  displayed  in  Table  HI. 


Table  HI.  Task  Success  of  Medium  Aptitude  Subjects  (40-50  Percentile)  Using  FPJPA 


Levels  of  Maintenance 

Problem 

Identification 

Non*Ts 

TS 

Percent  of 
Problems 
Successfully 
Completed 

O Level  Problems:  (12  subjects) 

19A2 

X 

90.5 

20  A- 1 

X 

91.1 

1 A 1 

X 

97.1 

4A2 

X 

57.5* 

FMS#2 

X 

97.0 

Composite  of  0 level: 

86.6 

I Level  Problems:  (23  subjects) 

6A4 

X 

X 

84.4 

PP6306 

X 

93.7 

4A2 

X(Align) 

68. 2a 

11  A3 

X(Align) 

82.5 

Composite  of  1 Level 

82.5 

Composite  of  0 and  I Levels 

84.1 

aProblem  required  each  subject  to  use  the  dual  beam  feature  of  the  oscilloscope.  Subjects  had  received  no  training 
concerning  this  feature  of  the  scope. 


These  results  are  only  slightly  lower  than  those  reported  for  the  AFHRL  advanced  development  study 
by  Potter  and  Thomas  (1976).  In  the  Potter  and  Thomas  study,  which  considered  only  troubleshooting 
problems,  the  inexperienced  subjects  isolated  89  percent  of  both  O and  I level  troubles  (see  Table  Fl, 
Appendix  F).  When  the  O and  I troubleshooting  problems  are  considered.  Table  Gl,  a composite 
percentage  of  about  88  percent  is  obtained  (87.7).  It  should  be  noted  that  the  Potter  and  Thomas  study 
utilized  high  aptitude  subjects  (80  percentile  and  above)  who  had  completed  36  weeks  of  conventional 
theory  based  training  whereas  this  study  utilized  only  average  aptitude  subjects. 
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7.  The  subjects  of  this  study,  as  well  as  those  in  the  Potter  and  Thomas  (1976)  study  and  the  Altus 
demonstration  (Mullen  & Joyce,  1974),  displayed  weaknesses  in  their  ability  to  use  key  test  equipment, 
especially  the  oscilloscope.  The  percentages  of  troubles  successfully  found,  although  impressive,  would  no 
doubt  have  been  higher  if  subjects  had  possessed  high  proficiency  in  the  use  of  their  test  equipment.  (Tire re 
is  also  substantia]  evidence  that  this  is  a common  weakness  of  many  DOD  experienced  maintenance 
technicians  (Foley,  1969).) 

Such  lack  of  test  equipment  proficiency  indicates  that  the  subjects  had  received  insufficient  “hands 
on”  practice  in  the  use  of  appropriate  test  equipment  functions.  In  the  Altus  effort  (Mullens  & Joyce, 
1974),  this  weakness  was  attributed  to  a lack  of  sufficient  practice  frames  in  the  training  package.  As  a 
result,  the  Altus  report  (Mullen  & Joyce,  1974)  recommended  that  future  TOT  packages  contain  more 
practice  frames.  Since  .he  TOT  packages  for  this  study  arrived  late,  most  subjects  had  insufficient  time  for 
completing  their  TOi  packages.  It  can  be  assumed  that  subjects  would  have  successfully  completed  even 
more  tasks  if  they  could  have  had  the  full  advantage  of  their  TOT  packages. 

However,  one  important  omission  was  found  in  both  the  O and  1 level  TOT  packages.  No  training  was 
provided  on  the  dual  trace  function  of  the  oscilloscope.  The  relatively  low  percentages  of  successful 
completions  for  the  O level  task  4A2  and  the  I level  task  4A2  (Table  HI)  were  the  direct  result  of  these 
inadvertent  omissions. 

8.  This  project  did  demonstrate  that  combining  the  FPJPA  and  TOT  technologies  was  feasible  and 
that  such  a tradeoff  would  reduce  initial  training  time  for  first  term  maintenance  personnel.  In  spite  of  all 
the  difficulties  previously  mentioned,  the  performances  of  the  medium  aptitude  subjects  of  this  study  were 
impressive.  Their  performances  were  as  good  or  better  than  those  of  high  aptitude,  conventionally  trained 
subjects  in  other  studies.  All  of  the  mentioned  difficulties  are  correctable  and  their  correction  would  result 
in  still  higher  levels  of  performance. 

Lessons  Learned 

1.  The  study  reiterates  findings  of  other  FPJPA  studies  that  the  effectiveness  of  quality  FPJPA  can 
be  substantially  degraded  by  users’  inability  to  use  their  test  equipments  proficiently.  In  all  future 
applications  of  FPJPA,  appropriate  actions  must  be  taken  to  assure  that  FPJPA  users  can  employ  their  test 
equipment  accurately. 

2.  Although  in-depth  “hands  on”  verification  procedures  were  only  applied  to  FPJPA  for  those  tasks 
used  in  the  evaluation,  a number  of  weaknesses  of  omission  and  commission  were  identified.  This 
verification  reiterates  the  requirement  for  the  review  and  acceptance  of  all  products  of  Tl&A(not  just  TS 
trees)  by  the  procuring  agency.  The  procuring  agency  must  also  ensure  that  the  appropriate  TI&A  products 
are  used  as  criteria  for  content  control  during  FPJPA  development.  And  these  assessment  activities  are 
necessary  for  quality  FPJPA  even  if  the  contractor  has  highly  experienced  personnel  in  FPJPA 
development.  (These  necessary  activities  require  that  the  procuring  activity  have  personnel  with  the 
necessary  skills.)  As  stated  in  the  body  of  this  report,  as  well  as  in  Appendix  B,  these  activities  together 
with  validation  and  verification  of  the  final  FPJPA  are  all  part  of  the  FPJPA  technology  and  are  necessary 
activities  for  ensuring  the  production  of  quality  FPJPA. 

3.  During  the  development  of  the  FPJPA  for  the  AN/AQA-7  systems,  its  1 level  maintenance 
concept  changed.  Originally  I level  maintenance  was  responsible  for  only  identifying  faulty  integrated 
circuit  cards,  all  within  card  repair  being  assigned  to  depot  maintenance.  This  within  card  responsibility  was 
transferred  to  the  I level,  approximately  tripling  the  number  of  1 level  trouble  signatures  for  FPJPA,  TS 
coverage.  Because  of  time  and  R&D  funding  limitations,  FPJPA  for  these  additional  signatures  were  not 
developed  for  this  project.  This  is  an  example  of  the  domino  effect  on  maintenance  guidance  and/or 
training  which  can  be  produced  by  a change  in  the  maintenance  concept  of  a system.  When  FPJPA  and 
TOT  are  involved,  the  results  are  immediately  apparent,  because  of  the  stringent  contant  controls  of  these 
technologies.  Although  the  effects  of  such  a drastic  change  in  maintenance  concepts  are  not  as  immediately 
apparent  with  traditional  theory  based  training  and  TMM,  the  results  no  doubt  are  hidden  in  increased 
destruction  of  parts  and  costly  secondary  damage  to  hardware-in  this  case  to  expensive  integrated  circuit 
cards. 
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APPENDIX  I:  ARMY  APPLICATIONS  01  IMG&l  AND  TOT  TECHNOLOGIES 


Although  the  Army  was  not  directly  involved  in  much  of  the  R&D  effort  concerning  either  improved 
maintenance  instructions  or  TOT  during  the  last  decade,  it  is  now  the  most  active  service  in  attempting  to 
exploit  these  technologies.  This  exploitation  has  had  several  stages.  The  first  was  a contract  effort  with 
KINTON,  Inc.  of  Alexandria,  Virginia,  which  resulted  in  a review  of  all  the  new  concepts  and  technologies 
concerning  improved  maintenance  instructions  and  TOT;  the  development  of  draft  specifications  for  job 
performance  manuals  (JPM)  and  job  performance  guides  (Draft  MU-M-632XX  (TM),  Part  1);  and  the 
development  of  an  accompanying  specification  for  TOT  (Draft  Mil-M-632XX  (TM),  Part  11). 

The  specifications  called  for  a TI&A  (called  “front  end”  analysis)  similar  to  that  found  in  the  Air 
Force  Vietnamization  specification  Mil-J-83302  and  AFHRL-TR-7343(I),  but  the  in-process  review 
requirements  of  the  products  of  Tl&A  for  Mil-M-632XX  were  not  as  stringent.  Directions  for  the  non-TS 
tasks  reflected  the  FPJPA  technology;  however,  the  verb  list  furnished  in  the  MU-M-632XX  omitted  the 
precise  definitions  found  in  Mil-J-83302.  Information  and  directions  for  TS  called  for  modification  of  the 
FORECAST  TS  aids  rather  than  the  FPJPA  technology.  The  major  modification  to  the  FORECAST  aids 
was  the  addition  of  pictorial  locators  for  test  points.  No  guidance  documents,  such  as  those  found  in 
AFHRL-TR-73-43  (Volumes  II  and  III;  Joyce  et  al.,  1973b,  1973c),  were  furnished  with  draft 
Mil-M-632XX.  Although  in  the  author’s  opinion,  the  FORECAST  TS  aids  are  not  as  effective  for  first 
enlistment  personnel  as  is  the  FPJPA  technology  for  TS,  there  are  substantial  hard  data  (Shriver  et  al., 

1964)  which  indicate  FORECAST  TS  aids  to  be  effective  with  such  personnel.  In  fact,  all  the  major 
technical  data  and  training  requirements  of  draft  Mil-M-632XX  (Parts  I and  II)  are  supported  by  substantial 
hard  data.  However,  the  FPJPA  technology  aspects  of  the  requirements  were  somewhat  “watered  down"  by 
the  less  stringent  in-process  review  requirements,  the  lack  of  guidance  documents,  and  the  removal  of 
precise  definitions  from  the  approved  verb  lists.  i 

This  KINTON  effort  produced  a Technical  Memorandum  29-75  (Shriver  & Hart,  1975),  which 
discusses  the  development  of  the  MU-M-632XX  specifications.  It  also  makes  some  predictions  as  to  the 
savings  to  be  expected  from  the  application  of  the  technologies  represented  by  these  specifications  in  terms 
of  shorter  initial  training  for  and  more  effective  operation  and  maintenance  performance  by  first  enlistment 
personnel. 

The  second  stage  of  the  Army  program  consists  of  the  application  of  Mil-M-632XX,  with  the 
exception  of  the  TS  aids,  to  three  systems;  the  Tank  Turret,  XM  1 Tank,  and  TACFIRE  (Tactical  Fire 
Direction  System).  For  these  systems,  LTTAs  of  uncontrolled  enrichment  are  replacing  the  recommended 
FORECAST  TS  aids.  There  are  no  hard  data  which  indicate  the  amount  of  LTTA  enrichment  required  to 
be  effective  for  first  enlistment  personnel. 

The  third  stage  of  this  program  has  been  the  replacement  of  draft  Mil-M-632XX  by  a series  of 
specifications  for  future  ITDT  projects,  including  Mil-M-63035  (TM),  -63036  (TM),  -63037  (TM),  -63038 
(TM)  and  companion  handbooks,  and  -63040  (TM).  The  Mil-M-63035  (TM)  retains  the  requirement  for 
“front  end”  analysis.  Mil-M-60340  (TM)  replaces  part  II  of  draft  Mil-M-632XX  concerning  TOT.  The 
remainder  of  the  series  provides  for  a number  of  technical  data  optimums.  The  effectiveness  of  none  of 
these  optimums  is  supported  by  hard  data.  The  JPM  for  non-TS  tasks  from  the  FPJPA  technology  has  been 
replaced  by  the  “new  look”  manual  which  contains  directions  supported  by  pictorials.  However,  the 
content  of  these  directions  is  not  in  standard  language.  Two  formats  of  LTTA  are  provided,  one  with 
pictorial  locators  and  one  without.  But  other  aspects  of  enrichment  are  uncontrolled  and  can  vary  from 
application  to  application. 
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The  fourth  stage  of  this  program  consists  of  the  application  of  this  modified  ITDT  including  “new 
look”  and  LTT A to  a number  of  systems  including  the  TOW/DRAGON  (anti  tank  missile),  the  Advanced 
Attack  Helicopter  (All-64),  the  M939(5T)  (5  ton  truck),  and  M204  Machine  Gun.  Since  these  modified 
ITDT  applications  contain  so  many  untested  and  watered  down  components,  their  effectiveness  for  first 
enlisted  personnel  will  probably  be  below  those  projected  in  Technical  Memorandum  29-75  (Shriver  & 
Hart,  1975),  as  well  as  below  what  could  be  expected  from  a full  application  of  the  I-'PJPA  technology 
coupled  with  TOT. 

The  fifth  stage  of  this  Army  program  calls  for  a revision  of  the  ITDT  specifications.  Such  a revision 
will  hopefully  correct  some  of  these  shortcomings  of  the  current  ITDT  and  thus  increase  their  potential 
effectiveness. 
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APPh’NDIX  J:  AIR  FORCE  APPLICATIONS  01  FPJPA  TECHNOLOGY 


r 


Most  of  the  Air  Force  maintenance  manual  efforts  reflecting  the  FPJPA  technology  have  been 
developed  under  Mil-M-38800A  which  reflects  only  the  format  aspects  and  not  the  content  control  aspects 
of  the  technology  provided  by  task  identification  and  analysis  (TI&A).  (TI&A  is  sometimes  called  "front 
end”  analysis.)  Such  applications  have  been  further  limited  to  the  0 level  of  maintenance  and  to  non-TS 
tasks.  However,  the  maintenance  manual  establishment  of  AFLC  has  taken  positive  action  to  include  the 
TI&A  aspects  of  the  technology  in  future  applications.  One  expected  outcome  of  the  recent  AFLC  Request 
for  Personnel  Research  (RPR)  77-l‘>,  mentioned  previously,  calls  for  the  development  by  Al  URL  of  a 
specification  for  maintenance  task  analysis  for  both  non-TS  as  well  as  TS  tasks  and  for  both  I and  O levels 
of  maintenance.  Another  promising  feature  of  this  action  is  that  the  developers  of  the  technology  are  being 
given  an  active  role  in  its  implementation. 

Of  the  past  Air  Force  maintenance  manual  improvement  efforts,  the  development  of  the  improved 
manuals  for  the  C-141  aircraft  most  nearly  reflects  the  full  scope  of  the  FPJPA  technology.  This 
development  applied  only  to  the  O level  of  maitnenanee.  The  requirements  for  the  TI&A,  as  well  as 
in-process  reviews  of  the  Tl&A  and  resulting  manual  development,  were  included  in  the  statement  of  work 
of  the  contract.  The  aids  called  Job  Guide  Manuals  (JCIMs)  for  the  non-TS  tasks  were  formatted  as  specified 
in  Mil-M-38800A. 

The  C-141  contract  called  for  the  development  of  LTTA  for  TS  tasks.  The  troubleshooting  or  logic 
trees  were  carefully  developed  by  the  contractor  based  on  hardware  analyses  which  identified  all  of  the 
trouble  signatures.  These  troubleshooting  trees  were  then  reviewed  for  their  adequacy  by  a group  of  Air 
Force  maintenance  specialists.  The  contractor  provided  a high  degree  of  enrichment  at  each  decision  point 
of  these  trees,  in  terms  of  directions  and  information.  Nouns  mentioned  in  the  directions  were  referenced 
to  locator  pictorials.  The  developmental  process,  as  well  as  the  enrichment  added  to  the  basic  logic  trees, 
was  very  similar  to  that  called  for  by  the  FPJPA  technology  for  TS  tasks. 

The  other  Air  Force  implementations  have  only  been  partial  applications  of  the  technology,  limited 
to  its  format  aspects  as  specified  by  Mil-M-38800A.  JGMs  in  this  format  have  been  developed  for  non-TS, 
organizational  maintenance  tasks  to  replace  existing  traditional  manuals  for  the  B-52.  KC-135,  and  the 
F-106  (partial).  Plans  are  to  update  other  systems  as  funds  become  available.  Such  JGMs  were  developed  as 
part  of  the  original  hardware  development  for  the  F-5L  and  the  A-10.  They  arc  being  developed  for  the 
F-16  and  arc  planned  for  the  Advanced  Medium  STOL  Transport  (AMST),  as  well  as  for  various  other 
subsystems  for  the  Air  Force  Avionics  Systems  Division  (ASD)  and  Electronics  Systems  Division  (ESD). 
JGMs  also  were  planned  for  the  B-l. 

As  to  support  of  TS  tasks  at  O level  of  maintenance,  the  Mil-M-38800A  authorizes  the  use  of  LTTA 
or  symptom-cause  charts  but  no  guidance  is  given  either  as  to  their  content  or  format.  As  stated  previously, 
symptom-cause  charts  provide  very  limited  TS  guidance.  The  organizational  maintenance  manuals  for  the 
A-10  include  JGM  for  non-TS  tasks  but  symptom-cause  charts  for  TS  tasks. 
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APPENDIX  K OFFICE  PRODUCTS  DIVISION  OF  IBM:  INTEGRATED  APPLICATIONS 
OF  IMG&l/TOT  TRADE-OFF  AND  JOB  TASK  PERFORMANCE  TESTS  (JTPT) 


The  Office  Products  Division  of  IBM  has  an  impressive  integrated  system  of  personnel  utilization.  It  is 
an  excellent  example  of  what  properly  applied  human  engineering  could  do  for  any  maintenance 
organization.  The  important  thing  for  DOD  to  emulate  is  the  process  and  its  organization.  The  exact  format 
of  this  IBM  personnel  utilization  system  was  designed  to  meet  the  needs  of  the  Office  Products  Division, 
wliich  of  course,  are  somewhat  different  from  those  of  DOD.  One  important  aspect  of  the  IBM  system 
which  could  be  emulated  by  DOD  is  that  success  in  all  training  and  job  advancement  is  based  primarily  on 
ability  to  perform  job  tasks  as  measured  by  task  performance  tests.  Some  other  important  aspects  of  this 
personnel  utilization  system  include: 

1.  Tire  responsibility  for  the  maintenance  manual  development,  service  ability  of  equipment, 
aptitude  test  development,  formal  training  development  and  execution,  field  training  packages,  and 
performance  assessment  center  are  under  one  directorate  the  Director  of  Technical  Operations. 

2.  The  selection,  hiring,  and  initial  training  of  maintenance  personnel  has  been  decentralized  and  is 
the  responsibility  of  the  field  service  managers.  However,  the  development  and  control  of  the  selection 
tests,  as  well  as  the  training  packages  are  the  direct  responsibility  of  the  training  establishment  under  the 
Director  of  Technical  Operations. 

3.  The  initial  training  packages  are  of  the  programmed  instruction  variety  and  provide  step-by-step 
exercises  for  "hands  on”  practice  of  key  maintenance  tasks  of  simple  IBM  office  equipment,  such  as 
typewriters  and  dictation  equipment  (IPE).  Successful  completion  of  this  training  program  is  dependent  on 
the  Trainee's  demonstrated  ability  to  perform  the  key  maintenance  tasks  of  these  equipments. 

4.  Graduates  of  the  entry  training  are  subject  to  an  extensive  battery  of  job  task  performance  tests 
given  in  a job  qualification  center.  From  1969  until  March  of  1977,  all  graduates  were  required  to  visit  an 
assessment  center  upon  completion  of  entry  training.  An  average  of  1,000  trainees  per  year  were  evaluated 
during  this  time  period.  This  performance  testing  program  has  had  two  objectives  - to  ascertain  the  quality 
of  entry  training  and  to  develop  a bank  of  performance  data  for  the  development  and  validation  of  the 
selection  test  battery.  (This  test  battery  was  validated  in  1973.)  Since  March  1977,  a modified  random 
sample  of  about  16  percent  of  the  entry  personnel  have  been  required  to  visit  the  assessment  center  for 
quality  control  purposes. 

5.  Once  the  employee’s  entry  field  training  is  completed,  very  careful  records  are  kept  concerning 
the  subsequent  job  success  as  measured  by  the  faults,  the  number  of  spare  parts,  and  the  time  required  for 
each  maintenance  action.  Selection  for  training  on  advanced  systems  is  based  on  job  success.  Training 
programs  for  advanced  systems  are  both  centralized  and  decentralized. 

6.  The  field  training  packages  for  initial  or  advanced  training,  as  well  as  the  centralized  training  for 
advanced  systems,  are  job  task  oriented.  The  training  materials  look  very  much  like  job  guide  manuals.  Over 
the  years,  if  the  task  directions  found  in  the  maintenance  manuals  were  deemed  adequate,  they  were  used 
for  training.  If  not,  task  directions  were  rewritten  for  training,  and  the  inadequate  directions  in  the 
maintenance  manuals  were  changed  accordingly. 

7.  This  IBM  integrated  personnel  utilization  system  including  training  and  maintenance  manuals  has 
been  the  result  of  years  of  such  "cut  and  try”  efforts  rather  than  on  a formal  TI&A  process.  This  is  possible 
when  the  number  of  types  of  equipment  is  relatively  small,  when  all  of  the  equipment  is  made  by  one 
manufacturer  (and  as  a result,  has  many  common  elements),  when  the  integration  is  directed  by  one 
manager  who  has  and  listens  to  excellent  human  factors  advice,  and  when  the  reaction  time  for 
implementation  of  changes  is  extremely  short. 


8.  Most  maintenance  directions  for  non-TS  tasks  are  given  in  step-by-step  format  supported  by 
pictorial  locators.  They  do  not  use  the  same  type  of  standard  language,  characteristic  of  the  FPJPA 
technology.  However,  they  do  use  a simplified  vocabulary  or  “common  English”  so  that  the  same  training 
materials  can  be  used  in  conjunction  with  IBM’s  world  trade.  There  are  two  types  of  TS  data;  one  is  very 
similar  to  the  format  called  for  by  AFHRL  fully  proceduralizcd  aids,  and  the  other  is  a flow  type  aid  similar 
to  that  used  in  FORECAST. 

9.  IBM  has  a functional  and  functioning  rotation  policy  for  managers  who  are  expected  to  have  staff 
experience  in  field  maintenance,  technical  support,  and  training.  All  of  the  managers,  including  vice 
preseidents,  started  their  careers  with  IBM  as  maintenance  technicians. 

10.  This  whole  integrated  personnel  utilization  system  is  the  result  of  a felt  need  on  the  part  of  IBM’s 
forward  looking,  and  maintenance  oriented,  management  in  the  middle  1960s.  IBM  was  going  to  expand 
the  number  and  complexity  of  its  office  products.  It  was  realized  that  to  do  this  profitably  their 
maintenance  organization  had  to  be  expanded  in  size  and  capability  with  a minimum  of  manhours  spend 
off  the  job  in  training.  (Even  with  this  expanded  number  of  systems,  many  single  Air  Force  Specialty  Codes 
encompass  a greater  number  of  systems,  which  reflect  the  design  eccentricities  of  many  manufacturers.) 


I 


60 


A 


APPENDIX  L:  BELL  SYSTEM  APPLICATIONS  OF  FPJPA  TECHNOLOGY 

The  Bell  System  uses  Tl&A  extensively  for  solving  operator  and  maintenance  personnel  and  training 
problems.  The  Bell  System  is  somewhat  similar  to  DOD  in  that  it  is  large  and  diverse  with  many 
more-or-less  independent  companies,  each  with  its  own  management.  It  is  different  from  DOD  in  that  its 
maintenance  technicians  (craftsmen)  have,  on  the  average,  much  longer  careers,  and  the  tasks  of  their  jobs 
are  not  as  diverse -Bell  System  having  only  one  source  of  hardware  i.e.,  Western  Electric. 

A reflection  of  the  strong  interest  in  Tl&A  by  Bell  System  was  its  sponsorship  of  a “Conference  on 
Uses  of  Task  Analysis  in  the  Bell  System”  in  October  1972  (Bell  System,  1972).  Approximately  40  papers 
were  presented  concerning  the  uses  of  TI&A  for  Job  Design  and  Restructuring,  Personnel  Research  and 
Administration,  Basic  Education,  Design  of  Systems,  Design  of  Training  and  Job  Performance  Aids.  The 
Bell  System’s  approaches  to  TI&A  have  the  same  foundation  and  are  very  similar  to  those  of  the  Advanced 
Systems  Division  (AS)  of  AFHRL.  The  Bell  users  of  TI&A  recognize  Dr.  Robert  B.  Miller,  who  gave  an 
invited  address  at  the  conference,  as  the  father  of  modern  task  analysis.  Dr.  Miller  accomplished  his  work  in 
this  area  at  the  American  Institutes  for  Research  (AIR)  under  Air  Force  contracts  in  the  1950’s.  The 
conference  proceedings  also  indicate  that  the  Bell  System  has  borrowed  heavily  from  Air  Force  reports  in 
its  development  of  Task-Oriented  Plant  Practices  (TOPP). 

Bell  has  also  utilized  both  TOT  and  JPA  technologies.  An  important  input  to  the  Bell  System  ongoing 
application  of  the  JPA  technology  was  an  internal  guide  for  JPA  development  by  Applied  Science 
Associates  (ASA)  of  Valencia,  Pennsylvania.  This  was  followed  by  a training  program  based  on  this  guide, 
given  to  Bell  personnel  at  Bell  Laboratories  by  ASA  personnel.  With  some  modifications,  this  training  is 
now  one  of  the  regular  and  popular  courses  given  by  Bell  Laboratory  at  Piscatway,  New  Jersey  for  Bell 
System  managers,  the  idea  being  to  greatly  reduce  or  eliminate  much  costly  training  by  using  good  job 
instructions. 

However,  Bell  System  does  not  have  as  formal  and  precise  trade-offs  of  TOT  and  JPA  as  do  some 
other  organizations.  The  applications  made  by  Bell  System  are  for  solving  specific  problems.  Some  of  the 
individual  Bell  Systems  are  larger  than  IBM  Office  Products  Division  and  little  information  seems  to  be 
available  concerning  specific  applications  by  individual  Bell  Systems.  Another  version  of  this  Bell  course  is 
given  by  New  England  Bell  at  Marlborough,  Massachusetts. 
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APPENDIX  M:  ARMCO  APPLICATIONS  OF  IMG&I  AND  TOT  TECHNOLOGY 


In  the  late  1960’s,  Armco  installed  electronic  controls  on  its  rolling  mills  in  place  of  previously 
installed  electro-mechanical  controls.  The  on-board  maintenance  technicians  were  having  considerable 
difficulty  maintaining  these  electronic  controls.  This  Armco  experience  is  reported  in  the  6 December  1976 
issue  of  Iron  Age  (Snodgrass,  1976).  The  problem  and  its  solution  contained  the  following  features. 

1.  Armco  management  had  foreseen  that  their  electro-mechanical  maintenance  technicians  would 
not  have  the  necessary  skills  for  maintaining  the  electronic  controls  and  had  attempted  to  solve  this 
deficiency  in  a traditional  manner.  For  several  years  an  apprentice  program  was  attempted.  This  program 
included  junior-college  level  theory  training  together  with  an  OJT  program  conducted  for  the  Armco  plant 
at  Butler,  Pennsylvania.  This  approach  was  tried  for  more  than  3 years.  But  the  products  of  this  training 
program  could  not  maintain  the  electronic  controls  in  a satisfactory  manner.  Finally  the  Armco 
management  engaged  ASA  to  help  solve  the  problem. 

2.  Based  on  TI&A,  ASA  developed  a JPA/TOT  trade-off  program.  Three  types  of  maintenance 
guidance  were  developed:  (a)  FPJPA  TS  aids  without  pictorial  locators.  (The  maintenance  technicians 
already  knew  the  geography  of  the  equipment);  (b)  a data  flow  chart  type  of  TS  aid  very  similar  to 
FORECAST  aids;  and  (c)  a preventive  maintenance  checkout  aid  indicating  normal  readings  at  all  check 
points  in  the  equipment.  A 1-week  programmed  instruction  package  was  developed  providing  “hands  on” 
practice  on  the  use  of  test  equipment  and  on  troubleshooting,  making  use  of  the  improved  maintenance 
guidance. 

3.  ASA  provided  a training  program  for  Armco  instructors  on  how  to  conduct  such  a trade-off 
program.  Armco  instructors  have  been  using  the  ASA  materials  to  train  their  own  maintenance  technicians 
on  an  ongoing  basis  using  one  instructor  for  two  students.  Armco  has  provided  actual  electronic  control 
equipment  for  the  required  “hands  on”  practice. 

4.  The  program  produced  immediate  improvements  in  the  performance  of  maintenance  tasks.  In 
addition,  the  use  of  the  preventive  maintenance  checkout  aids  made  it  possible  to  predict  many  component 
failures  in  advance,  so  that  they  could  be  replaced  during  scheduled  downtime,  thus  preventing  costly 
unscheduled  breakdowns. 

5.  Armco  has  developed  an  in-house  capability  for  developing  the  data  flow  charts  aids  for  other 
maintenance  applications. 
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Resulting  data 
used  to  standardize 
and  validate  improved 
aptitude  tests 


2-3  months 


Short  TOT  with 
supervised  practice  on 

1 Use  of  FPJPA 

2 Use  of  Hand  Tools. 

3 Use  of  Test  Equipment 

4 Safety  practices  (3) 


"4" 

Level 


First  Job 
Performed  with 

1 FPJPA  (Non-TS  Tasks) 

2 FPJPA  (TS  Tasks) 


1 Year 


Training  Gains: 

1 Length  of  initial  training 

reduced  as  much  as  75 
percent. 

2 Resulting  in  fewer  people 

requiring  training. 

3 Average,  as  well  as,  high 

aptitudes  could  be  trained. 

4 Shorter  personnel  build 

up  time  required  for 
national  emergencies. 

5 Reduction  in  initial  OJT 

and  job  cross  training. 

6 Reduction  or  elimination 

of  formal,  special  and 
supplemental  training. 


Maintenance  Gains: 

1 Such  personnel  job-ready  at  end  of  formal 

training. 

2 More  effective  performance  at  non-TS  tasks. 

3 More  effective  performance  of  TS  Tasks- 

estimate  such  personnel  can  isolate  90-95 
percent  of  possible  troubles,  (first  attempt) 

4 More  months  on  job  due  to  shorter  training 

time,  thus  fewer  personnel  in  the  DoO. 

5 Minimum  of  cross  training  to  unfamiliar 

equipments  (provided  FPJPA  available  on 
unfamiliar  equipments). 

6 More  effective  utilization  of  maintenance 

personnel  during  peak  and/or  emergency. 

7 Fewer  unsolved  maintenance  problems  refer- 

red to  depot  .'.  reduction  in  depot 
maintenance. 

8 Fewer  spare  parts  consurmed. 

9 Less  paper  work  generated. 

1 0  Less  secondary  damage  from  unnecessary 
maintenance  actions. 


I'igitre  2.  Annotated  flow  diagram  portr 
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First 

Term 


-/A- 


Remainder  of 
enlistment: 
perform  non-TS 
and  TS  tasks 
using  FPJPA. 

(10) 
•if-" 


a 

5 
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I I 

^ Two  additional 
I Opportunities  ^ 
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First 

Term 


A 


Second 

Term 


4 Years 


Broadbased  training 
developing  ability  for 
solving  and  correcting 
unusual  maintenance 
problems 
(12) 


~/h 


Job 

"1"  Level  Backup 
Specialist 

(13) 


-if- 


Job  Task  Guidance: 


Modified 

FOMM 

or 

SIMS 


1 FPJPA  (Non-TS  Tasks) 

2 FPJPA  (Most  TS  Tasks) 

For  Unusual  TS  Tasks: 

3 MDC  with  pictorial 

locators 

4 Blocked  schematics 

5 Keyed  information 

6 Pictorials  with 

locators  grids. 

7 Aircraft  wiring 

diagrams. 

8 IPB 

9 Preservation,  Shipping 

and  Storage  Into. 


Training  Gains: 

1 More  effective  training 

for  backup  personnel. 

2 Fewer  people  requiring 

long,  broadbased 
training. 


Maintenance  Gains: 

1 A larger  number  of  highly  effective  backup 

maintenance  personnel  for  performing 
unusual  and  difficult  maintenance  tasks. 

2 Fewer  unsolved  maintenance  problems  refer- 

red to  depot  reduction  in  depot 
maintenance. 


Second 

Term 


traying  a model  for  maximizing  benefits  of  F'PJPA,  SIMS  and  TOT  technologies. 


